Stivlo'st in Asia

Programming and Travel

Browsing Posts in Sysadm

Adding TRAC_ADMIN permission

No comments

By default in Trac authenticated users don’t see the Admin menu.

Here is how to enable it for the user stivlo of project sandbox. In the Trac console type the command permission add stivlo TRAC_ADMIN.

# trac-admin sandbox/
Welcome to trac-admin 0.12
Interactive Trac administration console.
Copyright (c) 2003-2010 Edgewall Software

Type:  '?' or 'help' for help on commands.

Trac [/var/projects/trac/sandbox]> permission add stivlo TRAC_ADMIN
Trac [/var/projects/trac/sandbox]> quit

Trac is an open source project management and task/bug tracking tool. It has also an integrated Wiki for project documentation. I’m using it since a few months already and I quite like it, it’s useful to keep myself organized. The installation procedure for version 0.12 is documented on Trac website.

I wrote this article to document how I’ve installed track myself, since there are a few possible choices and not all worked for me. For example the pip installation crashed with errors. I decided to use the default sqlite backend, which is fine since I mostly work alone and should be fine up to 10-20 concurrent users.

1) Install the needed tools

Download Trac and execute the following commands:

# apt-get install subversion libapache2-svn
# apt-get install python-setuptools apache2-mpm-prefork libapache2-mod-python
# tar xfvz Trac-0.12.tar.gz
# cd Trac-0.12
# python setup.py install

The install command will take care of dependencies, like the Genshi template system, and download them on the fly.

2) Prepare the folders

trac-admin will ask only two questions: one is the project name to which I replied ‘Playground’ and the other is the database connection string to which I replied just hitting enter to accept the default.

# mkdir -p /var/projects/trac/playground
# mkdir /var/projects/trac/tmp
# trac-admin /var/projects/trac/playground initenv
# mkdir -p /var/projects/svn/playground
# svnadmin create svn/playground --fs-type fsfs
# chown -R www-data /var/projects

3) Configure Apache

Next was to configure my Apache Virtual Host.

    #Where our trac projects are
    <Location "/projects">
        SetHandler mod_python
        PythonInterpreter main_interpreter
        PythonHandler trac.web.modpython_frontend
        PythonOption TracEnvParentDir /var/projects/trac
        PythonOption TracUriRoot /projects
        PythonOption PYTHON_EGG_CACHE /var/projects/tmp
        AuthType Basic
        AuthName "Trac"
        AuthUserFile /var/projects/projects.passwd
        Require valid-user
    </Location>

    #Require authentication for SVN access
    <Location /svn>
        DAV svn
        SVNParentPath /var/projects/svn
        AuthType Basic
        AuthName "Subversion Repository"
        AuthUserFile /var/projects/projects.passwd
        Require valid-user
    </Location>

4) Add WebDAV support and reload apache

The final step was to create the password file (/var/projects/projects.passwd) and add WebDAV module to apache and reload the configuration. At this point pointing the browser to the virtualhost /projects dir, the list of projects will show, containing only the playground project. Any further project can be added repeating step 2 with a different name and reloading apache.

# htpasswd -c /var/projects/projects.passwd myemail@example.com
# a2enmod dav_fs
# apache2ctl configtest
# apache2ctl graceful

This post doesn’t cover Trac-Subversion integration, they will both work but not connected to each others yet. Further it uses mod_python which has been recently discontinued (June 16, 2010). The preferred alternative is now mod_wsgi. I may look in the future how to install using mod_wsgi, but for now I don’t care since Trac works flawlessly and I’m not locked in mod_python, Trac project folder can be migrated any time in the future if it will be necessary to do so.

I copied the following text from S3tools-general mailing list and I’m posting it here because is an excellent explanation on how to enable S3 logs.

there are two sorts of access logs – one is the log of the S3 storage itself (generated per-bucket) and one is usage log for CloudFront, obviously only available if you deliver content from S3 over CloudFront, i.e. when you use S3 as a storage for web content as oppose to, say, for backup purposes.

The former (S3 logs) are enabled with:
s3cmd accesslog –access-logging-target-prefix=s3://your-log-bucket/target-bucket/s3/ s3://target-bucket
That will enable access logging for ‘s3://target-bucket’ and store the logs to ‘s3://your-log-bucket/target-bucket/s3/’ (obviously you should create s3://your-log-bucket first).

Similarly for CloudFront logging can be enabled with:
s3cmd cfmodify –access-logging-target-prefix=s3://your-log-bucket/target-bucket/cf/ cf://THEDISTID

At the end of the month simply download all the logs from s3://your-log-bucket/ and process to get whatever reports you need.

It is recommended to store the S3 logs in a separate bucket, not in the one being monitored. Otherwise writing the logfiles into the same bucket generates access requests and is logged too, which creates kind of a loop.

Hope that helps

Michal

Roundcube is a nice Ajax based webmail, but I was experiencing slow performance. I checked MySQL slow query log and I found queries taking several seconds. Expecially session and cache-update queries were taking as much as 9 seconds. This is really a long time for an interactive and fluid user experience. I’ve checked Roundcube version 0.3.1 and 0.4 and the tables are very similar. My hardware is a 3.0Ghz Xeon quad core server, so you’d expect something more snappy.

At first I checked MySQL configuration values, but I didn’t get the huge performance increase I was looking for. Then I checked the table structure: since where and order by fields where indexed already I was thinking to make the tables in fixed format. Unfortunately those two critical table included a blog field, that is variable length by definition so it’s not possible to achieve fixed length.

So I tried to optimize the fields as I could with the help of the Propose table structure function, but without going too close to the limits suggested, applying common sense and prudence.

The table cache had this structure:

CREATE TABLE cache (
 cache_id int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
 cache_key varchar(128)  NOT NULL ,
 created datetime NOT NULL DEFAULT '1000-01-01 00:00:00',
 data longtext NOT NULL,
 user_id int(10) UNSIGNED NOT NULL DEFAULT '0',
 PRIMARY KEY(cache_id),
 CONSTRAINT user_id_fk_cache FOREIGN KEY (user_id)
 REFERENCES users(user_id) ON DELETE CASCADE ON UPDATE CASCADE,
 INDEX created_index (created`),
 INDEX user_cache_index (user_id,cache_key)
)

cache_key is a varchar(128) string, while the maximum length in my db was 17 and it comes from a set of possibilities so it can’t possibly grow longer, except maybe in future version. I decided to set it to char(32), which is a fixed length type and so faster. It also means a smaller key file.

Also we have a data longtext field. A longtext is a data type able to contain up to 4Gb data! The maximum length in my table was actually 1517 bytes only. It’s a bit of overkill to use this type of field, so I changed it to text, a blob type that can hold up to 65535 and is simpler and faster.

I noticed a small speed increase, until I noticed that the table was a InnoDB table. The only reason I can think about to use InnoDB for a cache table is that “ON DELETE CASCADE ON UPDATE CASCADE” statement, since I don’t think there is any code updating user ids (would be crazy), the only use of that is being able to delete cache entry when a user is deleted. This small convenience comes at a high price! MyIsam tables are so much faster. So I removed the foreign key reference and change the table type to MyIsam.

I also noticed that I don’t have cached values older than about two weeks, even if I’m running the system since a few months, so I suppose cache gets cleaned periodically anyway and we don’t even need to clean expired user because will happen in two weeks time anyway.

That was the breakthrough improvement! Query time went under 1 second and didn’t appear in the query log again.

I applied the same concepts to session table and the query time when trying standalone queries went down 0.5s to 0.0005s! In the slow query log I had updates taking as much as 9s, but if they improve a thousand times too, no more query will appear in the query log.

To wrap up I think the performance improvement was amazing, but I will keep the slow query log checked for sometime before declaring this problem solved.

Update: solved the same problem with messages table. It seems there aren’t any slow queries now. :-)

As I recently moved to a new house, I don’t have my DSL subscription anymore and I decided to try to share the internet connection provided by my Nokia 5230 to the whole house using a wireless router (as a switch).

I’ve installed Ubuntu with pendrivelinux on a USB memory stick. It was fast and easy to connect to the internet and share it. After installing ubuntu I’ve installed wvdial:

# apt-get install wvdial

And it comes from the USB key itself, great thoughfulness because at this stage I can’t connect to the internet yet.

After that I’ve edited /etc/wvdial.conf

[Dialer Defaults]
Modem = /dev/ttyACM0
Baud = 9600
Phone = *99#
Username = user
Password = pass
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2
Init3 =
Dial Command = ATDT

Note that username and password can be anything in my case. By typing “wvdial” the internet connection is working, but only on this computer, is not shared yet. To share it, I wrote a small script, internet_share.sh

#!/bin/sh
echo "1" > /proc/sys/net/ipv4/ip_forward
iptables -A FORWARD -i eth0 -d 192.168.1.0/24 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -s 192.168.1.0/24 -o ppp0 -j ACCEPT
iptables -t nat -A POSTROUTING -o ppp0 -j MASQUERADE

After executing the script the connection is shared and will work, provided that other hosts of the network use the IP address of this computer as gateway. The connection is generally good and fast and I’m not missing my old DSL at all (that wasn’t that great actually).

Update: this configuration works with Smart Philippines, the configuration for Globe is a little different and I can post it if anyone is interested. I switched from Smart to Globe because Smart was stealing money from my account! Even if I was subscribed to a unlimited internet package, the account balance kept on shrinking, of several hundred pesos, despite I didn’t make any phone call or sent SMS. Globe doesn’t have this problem.

Powered by WordPress Web Design by SRS Solutions © 2010 Stivlo'st in Asia Design by SRS Solutions