Asset Publisher and Lucene indexes

As I wrote in previous posts here and here in latest version of Liferay there is performance loss around Asset Publisher and permission checking. We discussed that problem in LPS-22332 and we prepared a proof of concept based on concept that Lucene will be faster that plain DB queries. Just as we thought this was a good idea. Full document with results of our comparison tests you can read here.

Advertisement

About usability once again

Currently I’m testing Beta 3 of Liferay 6.1 release and despite many nice changes there are still couple of usability bugs.

New feature in Liferay 6.1 is that you can do almost anything with your site without using Control Panel. While it looks very nice and jazzy in my opinion it was bad thing to do as it often confuses users (especially if they know so much about CMS system as my mom does). They don’t know what is the difference between “Control Panel->Web Content->Add” web content and “Asset Publisher->Add new Basic Web Content” (as you may noticed there is inconsistency in those two labels). They just don’t know which of those two options they should use. Same goes for site settings and Edit Profile (this one opens layer even if you are in control panel already). IMHO most of those layers should be replaced by link that will redirect users to proper page in control panel. But lets forget for a moment about my feeling toward that and take a closer look at one aspect of those layers.

We have layers that do the same thing as Control Panel. After every change you will see Sucess message that will look like this:

Your request completed successfully. The page will be refreshed when you close this dialog. Alternatively you can hide this dialog.

As you may see “hide this dialog” is a nice link that will close your layer immediately. Now go to Control Panel, do the same thing. Ta daa, you will see the same message with the same link but this time it won’t do anything.

And now is the problem to solve. Should we remove that link? Should we duplicate messages for Layers and for Control Panel? Maybe we should leave it as it is and tell users that this is not a bug but a feature? Maybe we should replace layers with links to Control Panel and forget about such problems?

Well, I will leave that answer to you 🙂

Liferay leader in Garnter’s Magic Quadrant for Horizontal Portals

On 24th of October Gartner published results for his Magic Quadrant for Horizontal Portals. Once again Liferay is named a leader next to such big vendors like Oracle, Sap and Microsoft.  You can read whole article here.

As you can see below Liferay made a nice run for leader position.

Now the only challenge is to keep this good trend especially when there is such competitor like Drupal which has quite large community.  Also important is that Gartner pointed out couple of potential problems with fast grow of Liferay (you can read it here).
One of it is:

As it extends toward UXP, Liferay could become as bulky as its competitors and compromise its lean-portal value proposition.

This is the one we should all watch carefully because with very good improvement of look and feel of Liferay 6.1 came many UX mistakes and that can destroy all the advantages LR has over Miscrosoft Sharepoint or Oracle WebCenter. And where is new version of Social Office when we need it? Well, lets hope  all those failures will be corrected till final release which is scheduled for Q4’2011.

Till then, Congratulations Liferay 🙂

Asset Publisher and queries count

In this post, I will mainly describe my expierience with Asset Publisher in Liferay 6.1 (87489 revision). In the previous post I showed how Asset Publisher constructs entry query and how many records it picks up.

The starting point of my test was the following configuration:

• About 10 000 AssetEntries
• First page with 1 Asset Publisher without any filters. It gets ~4900 records.
• Second page with 1 Asset Publisher with categories filter. It gets ~ 300 records.

Following table shows results of my tests.

Scenario	Guest	Logged In user
1st request. Asset Publisher with ~4900 results. 20 per page.	~ 9800 queries to database	~ 29300 queries to database
2nd request. Asset Publisher with ~4900 results. 20 per page.	0 queries to database	~ 29300 queries to database
1st request. Asset Publisher with ~300 results. 20 per page.	~ 900 queries to database	~ 900 queries to database
2nd request. Asset Publisher with ~300 results. 20 per page.	0 queries to database	~ 30 queries to database

Interesting situation is the second row of this table (Asset Publisher with ~4900 results. 20 per page and logged in user). Asset Publisher retrievs 5000 records from database. Now in a loop it sends one query to transform AssetEntry to specific entity (JournalArticle in my case) and sends two queries to check permissions. It gives three queries in one record.

So Liferay sends 2 * (5000 * 3 queries per item) = 30 000 queries. In the trunk version the second query is cached so it gives 15 000 queries.

Fortunately, most of the queries is partially cached.

From a business point of view I want to display 20 articles in one AssetPublisher with pagination so I don’t understand why my portal sends over 30 000 queries to database. It’s a horrible situation when simple task frequently overuses my server resources.

One of bottleneck in Liferay 6.1

A few days ago we’ve prepared a jmeter’s test. Our goal was to find a bottlenecks in the newest Liferay 6.1 portal. So we load about 7k AssetEntries (JournalArticles only) and drag and drop one Asset Publisher on the site. To my surprise – page loaded for about 30 seconds. In my mind was born a strange thought: is it possible have we done something wrong? Maybe our content’s importer is killing the portal. So it’s time to debug Liferay.

In AssetEntryServiceImpl we found interesting code:

protected Object[] filterQuery(AssetEntryQuery entryQuery)
throws PortalException, SystemException {
[...]
int end = entryQuery.getEnd();
int start = entryQuery.getStart();
entryQuery.setEnd(end + PropsValues.ASSET_FILTER_SEARCH_LIMIT);
entryQuery.setStart(0);
List entries = assetEntryLocalService.getEntries(entryQuery);
[...]
}


We want to pick 20 entries and Liferay get 20 + 5000 records from database. Twenty – because AssetPublisher has to display this records, plus 5000 – just in case. It gives 10040 records per request in every Asset Publisher. Furthermore – Liferay calls this method two times in every request. Firstly to get countEntries (pagination), secondly to get entities.
Later in this method Liferay checks permissions on every entity and picks only entities which have View permission. I understand this logic, but it’s very expensive.

Maybe Liferay developers should find more optimal way to get Asset Entries. I leave this question open.

What is Liferay 6.1 release date

Answer is: today (today like in 21 september). At Wednesday we will see Liferay 6.1 EE and Marketplace. For those who can not attend Liferay West Coast Symposium there will be Live broadcast on Liferay.com/live so don’t miss it. We gonna prepare a lot of popcorn here at eo Networks.

Update: Well it look like it was only a presentation of upcoming version. betea release is scheduled for October 15 and stable release for Q4’2011.

Want to make Liferay even faster?

If your answer is yes then vote for issue LPS-14055. As you can see Asset Publisher makes many queries to underlying database. Those are very slow queries like count(*) (needed for pager element displayed below assets lists). If you are building big portal with many AP on every page (main article list, dynamic coin slider, recommended articles etc.) you will face lots of such queries. You can of course add additional servers to database layer but that is not a solution. Only thing that will help you is optimizing AP. First thing – in every query AP uses now() for checking expiration date. That lowers database abiliti to cache queries as with every second it gets new query. Second thing is cache – those queries are not made using hibernate. Third thing is speed – why ask database using very slow IN elements in queries if you can use JOINS. There is even better solution – use LUCENE for those slooooooow COUNT queries.

So if you want your Lfierays to become even faster vote for that particular issue now.

Do you know that you can not use characters like “:” in article titles?

As said here http://www.liferay.com/community/forums/-/message_boards/message/9002096 if you will use virtual host option in Liferay and add special character (like colon) Liferay will not translate it to proper page adress. Does any one knows is it fixed? Issue number maybe? A patch?

There are days where I feel the same thing as Liferay haters.

Liferay East Coast Symposium 2011

If you have any question find me on East Coast Symposium where I’m gonna give a speech about Liferay Community at 10th May 3:40 pm.

See ya.

Talking about integration

You all know that Liferay gives you many tools for quick integration with other application.

If it’s Java application you can deploy WAR file and with little luck you will see a new portlet with your app in Add application menu.
If it’s served as on separate page you can always show it in IFRAME portlet or in Proxy portlet.

You can also integrate you several applications using SSO, OpenID and LDAP.

But what if you want to always communicate to your users that they are still in your system?

For example let’s take combination of 3 apps: Liferay+OLAT LMS+Lime Survey. I know you can replace Lime Survey with Orbeon forms but here it serves another purpose – it’s not written in JAVA. First and easiest step is to share users through LDAP. In all those apps all is done by selecting proper option in control panel. Now let’s do SSO. Here you will ran into some problems as Lime Survey doesn’t have any CAS integration out of the box.  You can avoid that by selecting apps that support the same authorization solutions. And after that you will have nice pack of apps. But this not the way users see it.

Form their perspective those are different apps they have to learn to use. In most cases they will not even share same look and colors. And when they use their own window system and portlets and they check if it’s opened in IFRAME users will go nuts trying to go back to their Private Page (Liferay) from Learning Course (OLAT LMS). What you can do is to provide on all you apps somekind of unified “frequently used links” listing. And as you generally use Liferay the best way it will be to show it in the manner of Dockbar. Sounds interesting to you? If yes, than vote for ticket LPS-12281. Maybe it will be done as a part of “100 Paper Cuts” program.