Https not supported

I am currently trying to map all http pages to https pages. As the network providers are now investigating into giving advertisement based on your traffic. So I starting to encrypt all my traffic. While this is find quite a few pages that don't have https or map to http. The Google https start page for example just redirects to the http page. I don't really like this approach. Why don't the web page providers just enable https. I am aware that this will create a little bit more load on the servers but today there are many hardware encryption cards that will take away the load from the CPU. So I will still use Tor. When I get time I will start writing a crawler that checks a few websites if they enable https.

atime

Quite a few times now I had to time a program and take the average execution time. I normally did this with a little bash loop and then used a rough estimate or gnuplot to tell me the average runtime. Now I wrote a little python script that does exactly that. So you give it a number of loops and the program to execute.

$  ./atime.py 4 "sleep 0.5"
real 0.5
user 0.0
sys 0.0

for now you have to put the commands in quotes so it will pass in the whole command. Further be carefull not to loop to many times. Error catching works on a very rudenmental level.

$  ./atime.py 4 "xyz"
A error in the command has happend
/bin/sh: xyz: command not found

So if for your next assignment you need to take the average value of a program run time try atime.

 1 #!/usr/bin/python
 2 # A little tool that takes in a count and a command to
 3 # run and averages the execution time
 4 # ribalba@gmail.com
 5 
 6 import sys;
 7 import string;
 8 import os;
 9 
10 av_real = av_user = av_sys= 0
11 
12 looptimes = string.atoi(sys.argv[1])
13 
14 for i in xrange(looptimes):
15     commandout = os.popen3("time -p " + sys.argv[2])
16     sys.stdout.write ( commandout[1].read(),);
17     steddout = commandout[2].readlines()
18     if (len (steddout) > 3):
19         sys.stderr.write("A error in the command has happend\n")
20         sys.stderr.write(steddout[0] + "\n")
21         sys.exit()
22     av_real += (string.atof(steddout[0].split()[1]) / looptimes)
23     av_user += (string.atof(steddout[1].split()[1]) / looptimes)
24     av_sys += (string.atof(steddout[2].split()[1]) / looptimes)
25 else:
26   sys.stderr.write("real " + str(av_real) + "\n")
27   sys.stderr.write("user " + str(av_user) + "\n")
28   sys.stderr.write("sys " + str(av_sys ) + "\n")

[DOWNLOAD]

Cern Week 42

Most of the week was spent with the porting effort to SLC5. I tried fixing a bug in nscd that created an error when called two times after each other

$ /etc/init.d/nscd restart; /etc/init.d/nscd restart
Stopping nscd:                                             [  OK  ]
Starting nscd:                                             [  OK  ]
Stopping nscd:                                             [FAILED]
Starting nscd: /usr/sbin/nscd: already running
                                                         [FAILED]

This was due to a programming error in nscd that sent a SIGTERM to all its child processes but didn't wait for the return val. After quite a lot of research I just replaced the restart with a reload and it works. Further work went into the creation of SELinux rules so I don't get the horrible acv: denied errors in dmesg. Then I had to go to my Univeristy for the placement day. Which I will not comment on.

Another pain with google

I come to work and normally I start reading my emails and my rss feeds. As I already commented gmail is becomming soooo slow, so I started reading my mail in pine again. Now Google Reader stopped working. And my friends tell me that my photos on Picasa Web take about one hour to load. So I will be migrateing all my rss feeds to somewhere else. Goolge is stedely loosing me. Here a little break down what I used to use from google.

Mail => moved to pine : gmail to slow
Blog = > moving to wordpress : more functionality, better service, spellchecking works
Pictures => looking for something new : picasa to slow
Webpage => wordpress : Editor stopped working in Firefox
Rss Reader => looking for something new : Can't read my feeds
Calendar
iGoogle => switched off because of security conzerns
Code => back to sf : sf gives me ssh and a far better service

And quite a few more. So I used to have my whole life in google. And now I am noticing that I am moving away. A shame really google seamed such a nice company.

Looking for a replacement for gmail

I have been using gmail for quite some time now. And it is becoming worse by the month. Now they have a loading screen with a funky little loading bar. Besides it becoming so slow that a search for a word in all my mails takes about 10 seconds now I have to wait for it to load. I remember that's when I stopped using Thunderbird. So Google get your act together and remember 'every second counts'

Why open source does not work (sometimes)

Two friends and Me are currently starting a company. I can't say too much about the idea but it will be a traffic management online system. So I am off coding lots of nights. So the question was if we should open source the code. I normally publish everything I do under the BSD or GPLv3 license. I wouldn't say I am a Open Source activist but I try o keep my computer blob free. So now I am trying to sell a product. How would this work if it would be Open Source. Let's have a look at a successful open Source company that is not in the service business. So a company where 20 people sit in a room and code and someone gives them money for something they can have for free. Honestly I can not think of one company that sells a product and is open source. If you have a look at Google. They would never open source theire algorithm, if they would Microsoft would come along and build exactly the same search engine and give you 1 Cent if you search with it. Making money and open sourcing your code seam to be impossible if you are selling a product. It is brilliant if you are a service provider and you get your money from selling manuals or such, but if you are a little start up like us the code base is the only asset you have. Giving this away for free would ruin the whole project. So I feel quite sad to say that the next big thing I am coding will be closed source. Not because I believe in the closed source idea but because I will have to maybe live of this. And I can't see anyone giving us money for something that they could have for free.

While thinking about this I asked my self why is facebook, youtube and gmail the whole web 2 not Open Source. They all build on free software (mainly Linux I suppose) but still I would love to see the source code of gmail and find out why its sooooo slow. (See next post) The Google idea is that gmail will pay through adverts. But if there would be an open source version I would host it at home and not put up with that slow advert full version. So even a service has to be closed source sometimes to close in there customers to a product.

Cern Week 41

This week I spent most of the time with the migration effort to SLC5. This can be described in 3 main tasks.

- componenet porting
- certificate porting
- template porting

While porting the certificates I found a bug in openssl. If you did not specify any CApath it would not look into the standard location but if you set CApath to /dev/null if would find nothing in /dev/null and then look into the sandard certs direectory. Jan Iven sent a mail to Red Hat and hopefully this will be fixed upstream. I created a patch / workaround for the time beeing. Further we inforce template checking now. So I found loads of templates that did not conform with the appropriate schema. Further programs in SLC5 behave differntlelly as in SLC4. For example the autconfig tool does not set host and port anymore it uses the uri command. Stuff like this is quite annoying because you spend loads of time debugging something stupid like this. But finally we can run quattor and all the components I am responsible for on SLC5. Further zuul is now the default for all SLC5 so it has moved from "userspace" (the user includes pro_service_zuul) to the base tpl. So every quattor machine now uses zuul.

Cern Week 40

This week was spent partly at hepix
From hepix.org

The HEPiX forum unifies IT system support engineers from the High Energy Physics (HEP) laboratories and institutes, such as BNL, CERN, DESY, FNAL, IN2P3, INFN, JLAB, NIKHEF, RAL, SLAC, TRIUMF and others. The HEPiX meetings have been held regularly since 1991, and are an excellent source of information for IT specialists. That's why they enjoy large participation also from the non-HEP organizations.

Or I was preparing my presentation. Slides can be found here

Then on Thursday I gave my talk on 'The problem of managing 236 million user accounts'. This was a 30 min talk and I think it went quite well some people asked some interesting question so it was a success.

I listened to quite a few presentations but I didn't really learn as much as would have hoped. No further comments otherwise insults would be unavoidable.

The other main task for the week was to get good benchmark results of my implantation. I started looking for a time utility that would run a command n times and then give me the average run time but I could not find something like that. I had always done this with a horrible bash script, but this was now to inflexible as I could not pipe the output into a file that I then could parse with g
nuplot and so on. So I started the atime project. The basic idea is that you call it like time just with a number as first parameter. It will then run the command n times and output exactly like the standard *nix time utility would. Next post will have a full description. I wrote the tool in python because I didn't want to loose my python skills in favor of horrible Perl. I still don't fully understand why people use Perl for stuff like this. It's like hammering in a nail with the end of a screwdriver.

Amnesty International