Asset Publisher and Lucene indexes

As I wrote in previous posts here and here in latest version of Liferay there is performance loss around Asset Publisher and permission checking. We discussed that problem in LPS-22332 and we prepared a proof of concept based on concept that Lucene will be faster that plain DB queries. Just as we thought this was a good idea. Full document with results of our comparison tests you can read here.

About usability once again

Currently I’m testing Beta 3 of Liferay 6.1 release and despite many nice changes there are still couple of usability bugs.

New feature in Liferay 6.1 is that you can do almost anything with your site without using Control Panel. While it looks very nice and jazzy in my opinion it was bad thing to do as it often confuses users (especially if they know so much about CMS system as my mom does). They don’t know what is the difference between “Control Panel->Web Content->Add” web content and “Asset Publisher->Add new Basic Web Content” (as you may noticed there is inconsistency in those two labels). They just don’t know which of those two options they should use. Same goes for site settings and Edit Profile (this one opens layer even if you are in control panel already). IMHO most of those layers should be replaced by link that will redirect users to proper page in control panel. But lets forget for a moment about my feeling toward that and take a closer look at one aspect of those layers.

We have layers that do the same thing as Control Panel. After every change you will see Sucess message that will look like this:

Your request completed successfully. The page will be refreshed when you close this dialog. Alternatively you can hide this dialog.

As you may see “hide this dialog” is a nice link that will close your layer immediately. Now go to Control Panel, do the same thing. Ta daa, you will see the same message with the same link but this time it won’t do anything.

And now is the problem to solve. Should we remove that link? Should we duplicate messages for Layers and for Control Panel? Maybe we should leave it as it is and tell users that this is not a bug but a feature? Maybe we should replace layers with links to Control Panel and forget about such problems?

Well, I will leave that answer to you🙂

Liferay leader in Garnter’s Magic Quadrant for Horizontal Portals

On 24th of October Gartner published results for his Magic Quadrant for Horizontal Portals. Once again Liferay is named a leader next to such big vendors like Oracle, Sap and Microsoft.  You can read whole article here.

As you can see below Liferay made a nice run for leader position.

Now the only challenge is to keep this good trend especially when there is such competitor like Drupal which has quite large community.  Also important is that Gartner pointed out couple of potential problems with fast grow of Liferay (you can read it here).
One of it is:

As it extends toward UXP, Liferay could become as bulky as its competitors and compromise its lean-portal value proposition.

This is the one we should all watch carefully because with very good improvement of look and feel of Liferay 6.1 came many UX mistakes and that can destroy all the advantages LR has over Miscrosoft Sharepoint or Oracle WebCenter. And where is new version of Social Office when we need it? Well, lets hope  all those failures will be corrected till final release which is scheduled for Q4’2011.

Till then, Congratulations Liferay🙂

Asset Publisher and queries count

In this post, I will mainly describe my expierience with Asset Publisher in Liferay 6.1 (87489 revision). In the previous post I showed how Asset Publisher constructs entry query and how many records it picks up.

The starting point of my test was the following configuration:

  • About 10 000 AssetEntries
  • First page with 1 Asset Publisher without any filters. It gets ~4900 records.
  • Second page with 1 Asset Publisher with categories filter. It gets ~ 300 records.

Following table shows results of my tests.

Scenario Guest Logged In user
1st request.
Asset Publisher with ~4900 results. 20 per page.
~ 9800 queries to database ~ 29300 queries to database
2nd request.
Asset Publisher with ~4900 results. 20 per page.
0 queries to database ~ 29300 queries to database
1st request.
Asset Publisher with ~300 results. 20 per page.
~ 900 queries to database ~ 900 queries to database
2nd request.
Asset Publisher with ~300 results. 20 per page.
0 queries to database ~ 30 queries to database

Interesting situation is the second row of this table (Asset Publisher with ~4900 results. 20 per page and logged in user). Asset Publisher retrievs 5000 records from database. Now in a loop it sends one query to transform AssetEntry to specific entity (JournalArticle in my case) and sends two queries to check permissions. It gives three queries in one record.

So Liferay sends 2 * (5000 * 3 queries per item) = 30 000 queries. In the trunk version the second query is cached so it gives 15 000 queries.

Fortunately, most of the queries is partially cached.

From a business point of view I want to display 20 articles in one AssetPublisher with pagination so I don’t understand why my portal sends over 30 000 queries to database. It’s a horrible situation when simple task frequently overuses my server resources.

One of bottleneck in Liferay 6.1

A few days ago we’ve prepared a jmeter’s test. Our goal was to find a bottlenecks in the newest Liferay 6.1 portal. So we load about 7k AssetEntries (JournalArticles only) and drag and drop one Asset Publisher on the site. To my surprise – page loaded for about 30 seconds. In my mind was born a strange thought: is it possible have we done something wrong? Maybe our content’s importer is killing the portal. So it’s time to debug Liferay.

In AssetEntryServiceImpl we found interesting code:

protected Object[] filterQuery(AssetEntryQuery entryQuery)
throws PortalException, SystemException {
int end = entryQuery.getEnd();
int start = entryQuery.getStart();
entryQuery.setEnd(end + PropsValues.ASSET_FILTER_SEARCH_LIMIT);
List entries = assetEntryLocalService.getEntries(entryQuery);

We want to pick 20 entries and Liferay get 20 + 5000 records from database. Twenty – because AssetPublisher has to display this records, plus 5000 – just in case. It gives 10040 records per request in every Asset Publisher. Furthermore – Liferay calls this method two times in every request. Firstly to get countEntries (pagination), secondly to get entities.
Later in this method Liferay checks permissions on every entity and picks only entities which have View permission. I understand this logic, but it’s very expensive.

Maybe Liferay developers should find more optimal way to get Asset Entries. I leave this question open.

Tips & Tricks: include jsp into vm template

This tip&trick is dedicated to all developers who work with themes and vm’s. Probably you know that you have only little set of services from VelocityVariables class and some variables from init.vm. If you want call normal service method this is a huge problem. My question was: how can I include a jps file to vm file? Liferay has a lot of scriptlets and calls services from jsp. Why I can’t call it from VM? Many of you tell me: “I know how can you do it. Put this line of code and it’s ready”

#set ($categoriesService = $portal.getClass()
.getMethod("getService", null).invoke(null, null))

My answer is: Yessss, it works. But I want use a jstl lib (e.g. <aui> or <ui>) as well. In my opinion the simpliest way to do it is include jsp file to themeServletContext. Put your file.jsp into your theme:

and paste this piece of code into your vm (for example portal_normal.vm):

$theme.include($themeServletContext, "/path/to/your/jsp/file.jsp")

And enjoy all Liferay’s functionalities!

What is Liferay 6.1 release date

Answer is: today (today like in 21 september). At Wednesday we will see Liferay 6.1 EE and Marketplace. For those who can not attend Liferay West Coast Symposium there will be Live broadcast on so don’t miss it. We gonna prepare a lot of popcorn here at eo Networks.

Update: Well it look like it was only a presentation of upcoming version. betea release is scheduled for October 15 and stable release for Q4’2011.

Case study: East European Performing Arts Platform

The service has been made by eo Networks S.A. for the Adam Mickiewicz Institute (Instytut Adama Mickiewicza) –  ‘a state cultural institution whose task is to promote Polish culture around the world and actively participate in international cultural exchange’. The aim of the project was to create the platform for East European artists and institutions to share the information and cooperate in form of arts network. The core of the site is presentation of artists, events and cultural institutions.

Interesting features that we added to standard Liferay 6.0.3 CE are:

  1. Event Calendar (based on Web Content Structure and modified Asset Publisher)
  2. Advanced Journal Search so you can easily find events or artists
  3. Ability to attach files in web form portlet

The event structure enables editors to set the start and end date of events, such as theatre premiere, show, workshops etc. After that Event Calendar portlet will display current month and will mark days when events take place – start, end and all days in between. - main page - main page


Web form with attachement

Web form with attachement - event calendar - event calendar

Liferay and localization

With Liferay 6.0 you were unable to build news portals in different languages using mirror technique (“one web content in all languages”). This was huge bug for all those people who wanted to make portal outside English-speaking area. This disadvantage was corrected in Liferay 6.1 and now you can enter title and abstract in every language you want (from those in integrated in Liferay🙂 ). But what about other parts of Liferay, are those prepared for localization? Well answer is NO. Liferay still has long road to go to be fully prepared for such sites. For example you can’t localize notification emails (specially the one about registration in your portal).

So if you want this to be made quickly Vote for LPS-18394 (which was recently updated from “–Sprint – 08/11” to “Product Backlog”)

Want to make Liferay even faster?

If your answer is yes then vote for issue LPS-14055. As you can see Asset Publisher makes many queries to underlying database. Those are very slow queries like count(*) (needed for pager element displayed below assets lists). If you are building big portal with many AP on every page (main article list, dynamic coin slider, recommended articles etc.) you will face lots of such queries. You can of course add additional servers to database layer but that is not a solution. Only thing that will help you is optimizing AP. First thing – in every query AP uses now() for checking expiration date. That lowers database abiliti to cache queries as with every second it gets new query. Second thing is cache – those queries are not made using hibernate. Third thing is speed – why ask database using very slow IN elements in queries if you can use JOINS. There is even better solution – use LUCENE for those slooooooow COUNT queries.

So if you want your Lfierays to become even faster vote for that particular issue now.