[Back]

Startup Time Improvements (Part 1)
by cornelius, Sunday, October 19th @ 9:46 am

One thing that bothered me about our settings managers (and Compiz) was that they take too long to start. A major reason for this was the XML metadata loading and parsing. Settings managers (ccsm, simple-ccsm, and simple-ccsm-kde) do the metadata parsing through libcompizconfig, whereas Compiz does some metadata parsing itself and some through libcompizconfig when used with CCP (CompizConfig plugin).

Google recently opened up their data interchange format called Protocol Buffers, which has the potential to greatly speed up metadata loading for libcompizconfig and Compiz. So my plan was to load the metadata from binary Protocol Buffers files instead of from XML files in libcompizconfig and Compiz. The libcompizconfig part of that is now complete. The Compiz part will come in the future.

Protocol Buffers metadata parsing works like this: On the first run of Compiz or one of the settings managers, metadata is read from .xml files as before, and then written to binary .pb cache files under (UPDATE) $XDG_CACHE_HOME/compizconfig/ if $XDG_CACHE_HOME is defined and ~/.cache/compizconfig/ otherwise. On subsequent runs, metadata will be read from the .pb cache files resulting in reduced program startup time. If the locale has changed or if new plugins are enabled, the necessary .pb files will be updated at program startup.

Use of Protocol Buffers is enabled by default, but will be disabled automatically at compile time if the protobuf packages mentioned below are not installed. It can also be disabled explicitly at compile time with the -disable-protobuf configure option or at run time by setting the COMPIZ_NO_PROTOBUF environment variable to 1, yes, or true.

Aside from using Protocol Buffers for metadata, there were other changes that improved program startup time, namely removal of global metadata used by libcompizconfig and loading of plugin icons in the background when ccsm starts.

So the overall startup time improvements compared to Compiz Fusion 0.7.8 (at least on my system) are as follows:

Simple-CCSM: starts about 4.5x as fast (78% less time)
CCSM: starts about 2.4x as fast (58% less time)
Compiz (when used with CCP): starts about 1.25x as fast (20% less time)

When Protocol Buffers metadata parsing is implemented in Compiz as well, you will see more startup-time improvement for Compiz. Also, note that these results are for hot startup, that is, not for the first run after a reboot. Cold startup time improvements will vary.

To get the full speedup while testing the changes, make sure you install the protobuf packages as explained below and install the latest git versions of libcompizconfig, compizconfig-python, ccsm, and compiz. NOTE: You may need to do “make distclean” before compiling libcompizconfig.

If you run into any problems, please let us know by filing a bug report at http://bugs.opencompositing.org/.

How to install Protocol Buffers (to get the full speedup)

Ubuntu/Debian users should install the packages libprotobuf0, libprotobuf-dev, and protobuf-compiler. For Ubuntu 8.10 (Intrepid) and Debian-unstable, they are available in the official repositories. For Ubuntu 8.04 (Hardy) they can be downloaded from
http://ppa.launchpad.net/davidf/ubuntu/pool/main/p/protobuf/

or can be installed after adding the following lines to /etc/apt/sources.list and doing an apt-get update:
deb http://ppa.launchpad.net/davidf/ubuntu hardy main
deb-src http://ppa.launchpad.net/davidf/ubuntu hardy main

OpenSUSE, Fedora, and Mandriva users should install the protobuf and protobuf-devel packages. If not available in the official repositories, they can be found at
http://download.opensuse.org/repositories/home:/LenzGr

Users who want to install from sources can find the latest Protocol Buffers sources at:
http://code.google.com/p/protobuf/downloads/list

For packagers: On Ubuntu and Debian only the libprotobuf0 package is needed at run time, so no need to make the protobuf-compiler or libprotobuf-dev packages run-time dependencies for libcompizconfig. Similarly, on OpenSUSE, Fedora, and Mandriva only the protobuf package is needed at run time.



6 Comments »
TrackBack URL

  1. : Dieter_be, October 19 @ 2:12 pm

    Good idea!
    How do you check the validity of the cache? mtime on the xml? md5sums?..?

    Also,I think it would be better to store the cache data in $XDG_CACHE_HOME/compiz/compizconfig/cache instead of using $XDG_CONFIG_DIR
    ( See http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html )
    People using version control on their ~ (and in general everyone who likes a clean ~ ) would be very thankful for that

  2. : cornelius, October 19 @ 2:19 pm

    @Dieter_be: Yes, I look at the modification time of xml files. And thanks for the feedback. I’ll consider that.

  3. : cornelius, October 19 @ 4:06 pm

    @Dieter_be: Cache directory path is now changed to conform to the standards you mentioned.

  4. : Compiz-Fusion を最新版(ver.0.7.9 git master)に入れ替え « Kawaji’s Weblog, October 29 @ 7:38 am

    [...] 1つ目は Compiz-Fusion の設定ファイルの読み込みのスピードアップ化を目的とした変更に伴うもの。 Compiz-Fusion 公式サイトのブログにその説明があるんですが、今までは設定内容をXMLファイルから読み込んでいたけれど、Compiz-Fusion の最新開発版ではそのXMLファイルを Google が最近オープンソース化した Protocol Buffers と呼ばれるバイナリ形式の新しいフォーマットのファイル(拡張子: .pb )に一旦変換して、その pb ファイルから読み込むようにしたそうです。 これによって例えば Simple-CCSM の起動が 4.5 倍、CCSM の起動が 2.4 倍くらい速くなるとか。 コンパイル時に自動的に protobuf パッケージがインストールされているかチェックされるため、protobuf がインストールされていない場合は以前の通りにXMLを読み込む方式を取るようにコンパイルされるだけで、エラーを吐いてコンパイルに失敗するということはないみたいです。 PB を利用できるように Compiz 最新開発版ソースをコンパイルするには libprotobuf0 と libprotobuf-dev および protobuf-compiler の3つのパッケージをインストールしておく必要があるそうです。 Ubuntu 8.10 (Intrepid) では公式リポジトリから普通に入手可能だそうですが、Ubuntu 8.04 の場合はこちらからダウンロードするか、以下のAPTラインを新たなリポジトリとして追加します。 deb http://ppa.launchpad.net/davidf/ubuntu hardy main deb-src http://ppa.launchpad.net/davidf/ubuntu hardy main [...]

  5. : Compiz Fusion Community News for Nov 4, 2008: Tons of new developments! « SmSpillaz’ Blog, November 4 @ 6:41 am

    [...] you haven’t read Erkin’s post about improved metadata parsing optimisation in compiz-fusion, you should definitely take a [...]

  6. : khanhpt, November 11 @ 7:32 am

    I run fusion-icon in terminal -> error:

    File “/usr/lib/python2.5/site-packages/FusionIcon/util.py”, line 23, in
    import os, compizconfig, ConfigParser, time
    ImportError: /usr/local/lib/libcompizconfig.so.0: undefined symbol: _ZN6google8protobuf14DescriptorPool26InternalBuildGeneratedFileEPKviPFvPKNS0_14FileDescriptorEE