Not Really a Blog

January 26, 2007

Firefox trick

Filed under: Computers, Internet — Tags: , , — jesus @ 10:53

Try this in firefox:

  • Go to any page with lots of images.
  • Delete the URL bar
  • Substitute it with:
javascript:R=0; x1=.1; y1=.05; x2=.25; y2=.24; x3=1.6; y3=.24; x4=300;y4=200; x5=300; y5=200; DI=document.images; DIL=DI.length; function A(){for(i=0; i-DIL; i++){DIS=DI[ i ].style; DIS.position='absolute'; DIS.left=(Math.sin(R*x1+i*x2+x3)*x4+x5)+"px"; DIS.top=(Math.cos(R*y1+i*y2+y3)*y4+y5)+"px"}R++}setInterval('A()',5); void(0)

Press Enter and enjoy!!!

:-)

December 13, 2006

More on setting a subversion mirror repository

Filed under: Internet, Programming — Tags: , , , — jesus @ 22:34

A few days ago I wrote about setting up a subversion repository using svnsync. On that entry I was using svn+ssh authentication, but there are some gotchas to take into account to avoid having a security issue.If you recall correctly, we had a unix user with which people will access the repository. If we haven’t taken any extra protection, that means that anyone that has their ssh key on such a user’s ~/.ssh/authorized_keys file can log into the system. If such user uses public authentication to access the remote account used for the remote repository, any committer can have access to the private key and thus access the remote machine and have write permissions on it.

To avoid this scenario we need to set things up in a way that users can commit to the main repository but cannot have access to the remote repository. And that, again, can be done by creating a different user (Remember, all this applies to the main repository box, not the remote box).
Say:

  • svn: The user used for the repository. It will let any user have write permission on the repository. It will also have all the public keys of all the committers, to allow them to access the server using the svn+ssh authentication under the generic user svn, in case it needs to be retrieved from a remote location, which can be easily used by forwarding your credentials.
  • svnsync: A user which will have access to the repository (read access is enough) and will have a couple of wrapper scripts to svnsync. It will use a ssh key to access the remote repository. The rest of the users will have no read permissions on this user’s home directory.
  • The rest of the users for the committers. They will have write access to the repository. As you may know, if you use svn+ssh authentication, whenever you do a commit, you basically are executing svnserve on the repositoy under your user ID (That’s why you need write permissions).

This way, committers won’t have access to the svnsync private key. But we must grant them access to the wrapper scripts, so whenever they do a commit, svnserve is able to execute those scripts by triggering a post-commit hook.

All these can be achieved by using sudo. So, say that we have two different wrapper scripts:

  • /home/svnsync/bin/synccommit.sh
    #!/bin/bash
    # this is to synchronize the repostory
    # $1 will be the repository
    if [[ -z $1 ]]
      then
        # The repository should be a parameter
        exit
    fi
    
    DATE=`date +%Y%m%d-%H%M `
    
    echo "[$DATE] Commiting to repository $1 . (output below if any)" >> /home/svnsync/svnsync-commit.log 2>&1
    /usr/local/bin/svnsync synchronize --username svnsync "$1" >> /home/svnsync/svnsync-commit.log 2>&1
    
  • /home/svnsync/bin/syncprop.sh
    #!/bin/bash
    # this is to synchronize the a property
    
    # $1 will be the repository
    # $2 will be the revision number
    
    if [[ -z $1 || -z $2 ]]
      then
        # The repository should be a parameter
        exit
    fi
    
    DATE=`date +%Y%m%d-%H%M `
    echo "[$DATE] Updating property (Revision $2) to repository $1 . (output below if any)" >> /home/svnsync/svnsync-properties.log 2>&1
    /usr/local/bin/svnsync copy-revprops --username svnsync "$1" "$2"  >> /home/svnsync/svnsync-properties.log 2>&1
    

We can call them by setting up the hook scripts as:

  • post-commit hook:
    # Propagate the data to the remote repository
    sudo -u svnsync /home/svnsync/bin/synccommit.sh svn+ssh://svnsync@remote/home/svnsync/svn &
    
  • post-rev-change:
    # Propagating changes to the remote repository. Putting it to the background
    sudo -u svnsync /home/svnsync/bin/syncprop.sh svn+ssh://svnsync@remote/home/svnsync/svn $REV  &
    

    All we need to do now is grant access to the committers to execute both wrapper scripts by adding the following to the /etc/sudoers file (remember, by using visudo)

    %commiters  ALL= (svnsync) NOPASSWD: /home/svnsync/bin/syncprop.sh
    svn         ALL= (svnsync) NOPASSWD: /home/svnsync/bin/syncprop.sh
    
  • Where committers is a common group where all committers belong to.

    December 4, 2006

    Speeding up trac’s response time

    Filed under: Internet, Programming — Tags: , , , , — jesus @ 18:30

    I’ve been trying to speed up an installation of trac over the last few days. The web interface took ages to display each of the directories or files within the subversion repository. But this one wasn’t too big. The only change to the subversion repository is that we started using a vendor branch imported into our main repository using svm

    So, after a few hours trying different solutions, and reading trac’s source code, I think I got where the bottleneck was.Well, it was http://www.sqlite.org/download.html which was causing the bottleneck. Trac uses an object CachedRepository to access the repositories. Whenever we want to get the chagesets, a function to synchronize the repository is called:

    class CachedRepository(Repository):
      def get_changeset(self, rev):
        if not self.synced:
          self.sync()
          self.synced = 1
          return CachedChangeset(self.repos.normalize_rev(rev), self.db, self.authz)
    

    and such method, sync(), makes a call to:

    youngest_stored = self.repos.get_youngest_rev_in_cache(self.db)
    

    which is all this:

    def get_youngest_rev_in_cache(self, db):
        """Get the latest stored revision by sorting the revision strings
        numerically
        """
        cursor = db.cursor()
        cursor.execute("SELECT rev FROM revision ORDER BY -LENGTH(rev), rev DESC LIMIT 1")
        row = cursor.fetchone()
        return row and row[0] or None
    

    And that SQL query was taking around 1-2 seconds each time it was executed. It happened that we were running an old version of sqlite and pysqlite, so a ./cofigure && make && make install using the recommended installation saved my day :-)

    Hope it is useful to anybody if it gets indexed by Google.

    November 30, 2006

    Setting up a subversion mirror repository using svnsync

    Filed under: Internet, Programming — Tags: , , , — jesus @ 01:48

    With the new version of subversion 1.4 you have a new tool called svnsync with which you can maintain mirror repositories quite easily. I’ve been working on one at work and would like to share with you my findings in case it interests anyone :-)

    Understanding the problem

    In order to have a mirror repository, it is important that commits only happen on the master and then they are synchronized to the mirror using the svnsync program. Then, the mirror repository can be used for whatever you may think of but for committing: backup, high speed checkouts, web front-ends, etc.

    So, svnsync must be the only “one” able to commit to the repository. If we use the apache integration, there are various ways to do this. Let’s say we are using svn+ssh for authentication, in which case it is more complicated as ssh access usually grants writing access to the file system. So creating a different user is going to be handy.

    Creating and populating the repository

    So, let’s say that we created a user called svnsync on the target machine and that we are going to create a new subversion repository in its home directory:

    svnadmin create /home/svnsync/svn

    Now, we need to set up a hook to let svnsync change the properties. For this, we create /home/svnsync/svn/hooks/pre-revprop-change with:

    #!/bin/sh
    USER="$3"
    
    if [ "$USER" = "svnsync" ]; then exit 0; fi
    
    echo "Only the svnsync user can change revprops" >&2
    exit 1
    

    We will grant access to the user running svnsync on the main machine by copying its ssh key to .ssh/authorized_keys. And now, we only need to initialize the remote repository. Note that we can run svnsync from where ever we want, but for the sake of simplicity, we will run it on the main machine, where the original repository resides.

    $ svnsync init --username svnsync \
          svn+ssh://svnsync@remote/home/svnsync/svn \
          file:///source-svn
    

    Note:

    • The syntax is
      svnsync init DEST SOURCE

      That’s it, the destination repository goes before the source repository.

    • There is no “:” between the host name and the path to the remote repository.

    With this command, we will have initialized the destination repository and now we are ready to populate the destination repository. We can do it with this command:

    svnsync synchronize --username svnsync \
           svn+ssh://svnsync@remote/home/svnsync/svn
    

    And, as we already initialized the repository, there is no need to specify the source repository. This will take more or less time depending on how big you repository is and how fast your network connection is. Hopefully it will have finished after you take a coffee :-)

    Creating another user to let users access the repository

    So, we will now create an user called svn which will be used to access the repository using the subversion client. As we are using svn+ssh, all we need is to grant access to such user to all the users that have access to the main repository. If we are using ssh keys it’s as easy as copying all the allowed keys to the /home/svn/.ssh/authorized_keys file.
    Also, if we change the permissions to the repository at /home/svnsync/svn (and its parent) to be something like

    drwxr-xr-x  7 svnsync users 4096 2006-11-28 17:30 svn/

    we will let svnsync (and the svnsync user) be the owner and have write permissions to the repository and let svn (and all the users ssh’ing) have read access only to the repository (provided both belong to the users group).

    $ svn co svn+ssh://svn@remote/home/svnsync/svn/test
    A    test/trunk
    [...]
    A    test/trunk/readme
    Checked out revision 2.
    $ echo "test" >> test/trunk/readme
    $ cd test/
    $ svn -m "test commit" ci
    Sending        trunk/readme
    Transmitting file data .svn: Commit failed (details follow):
    svn: Can't create directory '/home/svnsync/svn/db/transactions/2-1.txn':
    Permission denied
    

    And that’s all.

    Commiting to the master respository

    In case you want to commit back to the master respository, you need to do a “svn switch –relocate” to point to the master repository, but for that to work, it needs to have the same UUID if we don’t want it to fail.

    1. To get the UUID on the main machine:
      svnadmin dump -r0 /source-svn | head -n 3 > saved-uuid
      
    2. Copy the file saved-uuid to the remote machine and do a
      svnadmin load --force-uuid /home/svnsynd/svn < saved-uuid
      

    So, thins to take into account:

    1. When populating the repository, we use the svnsync user who has write permissions to the repository (svn+ssh://svnsync@…)
    2. When checking out, we use the svn user (svn+ssh://svn@…)
    3. Automatization

      If we want to keep both repositories synchronized, we need to set up a couple of hooks on the source repository.

      post-commit

      Add this to such hook (it needs to be executable and not have the .tmpl extension)

      # Propagate the data to the remote repository
      /usr/local/bin/svnsync synchronize --username svnsync
              svn+ssh://svnsync@remote/home/svnsync/svn &
      

      post-rev-changes

      We also want the properties changes to be present at the remote repository:

      # Propagating changes to the remote repository.
      /usr/local/bin/svnsync copy-revprops --username svnsync
             svn+ssh://svnsync@remote/home/svnsync/svn $REV  &
      

      Note that we put them to the background to let the user go on with what he was doing.

      Final notes

      This is a quick guide on how to set things to have a remote repository. There is much more than this and I recommend you to read the documentation and, obviously, do a backup. Doing a “svnadmin dump” only takes a while a it´s really worth it.

      In any case, just let me know if you find any errors or typos.

    October 3, 2006

    Mini tutorial on gdb

    Filed under: Programming — Tags: , , , — jesus @ 02:43

    Thanks to Matthew:

    gdb --args program/to/debug --whatever
    break Foo.c:532
    # to put a breakpoint at line 532 of Foo.c
    run # to run it, until it hits that line
    bt # to display the stacktrace (backtrace)
    frame 2 # to change the stack frame you're looking at
    list # to list the sourcecode for that stack frame
    step # to step into the next command for that frame
    next # to run the code to the next command
    continue # to continue onwards
    print bar # to print the contents of variable bar
    printf  "%08x", bar # standard printf for bar
    delete 2 # to remove the 2nd breakpoint
    

    Obviously, you need to compile things with -g as an argument to GCC.

    The Shocking Blue Green Theme. Blog at WordPress.com.

    Follow

    Get every new post delivered to your Inbox.

    Join 2,867 other followers