systemcall dot org

Unix is turning 40 and BBC folks wrote a small article about it. What a joy to remember when I started using Unix (AIX on an IBM machine) around 1994 and was overwhelmed by it.

By that time, the only Unix that ran well on a PC was SCO and that was a fortune, but there were some others, not as mature, that would have the same concepts. FreeBSD and Linux were the two that came into my sight, but I have chosen Linux for it was a bit more popular (therefore could get more help).

The first versions I’ve installed didn’t even had a X server and I have to say that I was happier than using Windows. Partially because of all the open-source-free-software-good-for-mankind thing, but mostly because Unix has a power that is utterly ignored by other operating systems. It’s so, that Microsoft used good bits from FreeBSD (that allows it via their license) and Apple re-wrote its graphical environment to FreeBSD and made the OS X. The GNU folks certainly helped my mood, as I could find all power tools on Linux that I had on AIX, most of the time even more powerful.

The graphical interface was lame, I have to say. But in a way it was good, it reminded me of the same interface I used on the Irix (SGI’s Unix) and that was ok. With time, it got better and better and in 1999 I was working with and using it at home full time.

The funny thing is that now, I can’t use other operating systems for too long, as I start missing some functionalities and will eventually get locked, or at least, extremely limited. The Mac OS is said to be nice and tidy, and with a full FreeBSD inside, but I still lacked agility on it, mainly due to search and installation of packages and configuration of the system.

I suppose each OS is for a different type of person… Unix is for the ones that like to fine-tune their machines or those that need the power of it (servers as well) and Mac OS is for those that need something simple, with the biggest change as the background colour. As for the rest, I fail to see a point, really.

After the buzz on Windows 7 I decided to take a look on a video posted (apparently by Microsoft itself) on youtube.

I was expecting to hear about the new Operating System, only to find out that everything that matters to MS is the Window Manager. No memory or CPU consumption reports, no filesystem or network configuration structure, nothing.

Anyway, I have to talk about the interface then… Now, is it only me or they’re copying what Gnome/Compiz is doing? Because, copying Apple it’s obvious, even Gnome/Compiz is at some extent.

First, window transparency and ALT-TAB with window thumbnails to select with your mouse. Done. Second, dragging icons to the taskbar, what’s new on THAT?!

But now, “something really cool we’re putting in Windows 7 is called ‘snap-to’ “?!?!? If I recall right, the graphical interface from the PARC team already had tiled/cascade window arrangement and undoubtedly Microsoft used that on Windows 3, so how is that cool in any way?

Well, they better have a much stabler environment and much lower footprint, otherwise they won’t have nothing really serious to show.

It still hasn’t gone to meet it’s maker, but it was also not as bad as it could’ve been.

After Windows Vista was launched with more PR and DRM than any other, Microsoft hoped to continue its domination of the market. Maybe afraid of the steep Linux increase in desktops (Ubuntu has a great role in that) and other market pressures, they’ve rushed out Vista with so many bugs and security flaws, so slow and with such a big memory and CPU footprint that not many companies really wanted to change their whole infrastructure to see it drawn a little later.

China government ditched it for XP because it was not stable enough to run the Olympics, only to find out that the alternative didn’t help at all.

All that crap helped a lot Linux (especially Ubuntu) jump on the desktop world. Big companies shipping Linux on lots of desktops and laptops, all netbooks with Linux as primary option, lay people now using Linux as they would use any other desktop OS. So, is it just because Vista is so bad? No. Not at all. Linux got really user friendly over the last five to ten years and it’s now as easy as any other.

Vista is so bad that Microsoft had to keep supporting Windows XP, they’re rushing again with Windows 7 and probably (hopefully) they’ll make the same mistakes again. It’s got so bad that the Free Software Foundation’s BadVista campaign is officially is closing down for good. For good as in: Victory!

Yes, victory because in one year they could show the world how bad Vista really is and how good the other opportunities are. Of course, they were talking about Linux and all the free software around, including the new gNewSense platform they’re building, but the victory is greater than that. The biggest message is that Windows is not the only solution to desktops, and most of the time, it’s the worst.

In conjunction with the DefectiveByDesign guys, they also showed how Vista (together with Sony, Apple, Warner et al) can completely destroy your freedom, privacy and entertainment. They were so successful in their quest that they’re closing doors to spend time (and donors’ money) in more important (and pressing) issues.

Now, they’re closing down but that doesn’t mean that the problem is over. The idea is to stabilise the market. Converting all Windows and Mac users to Linux wouldn’t be right, after all, each person is different. But the big challenge is to have users that need (or want) a Mac, to use a Mac. Who needs Windows and can afford to pay all extra software to protect your computer (but not your privacy), can use it. For developers the real environment is Unix, they should be able to get a good desktop and good development tools as well. It’s, at least, fair.

But for the majority of users, what they really want is a computer to browse the web, print some documents, send emails and for that, any of the three is good enough. All three are easy to install (or come pre-installed), all three have all the software you need and most operations and configurations are easy or automatic. It’s becoming more a choice of style and design than anything else.

Now that Apple got rid of all DRM crap, Spore was a fiasco so EA is selling games without DRM, the word is getting out. It’s a matter of time it’ll be a minor problem, too. Would DefectiveByDesign retire too? I truly hope so.

As an exercise to the reader, go to Google home page and search for the terms: “windows vista“. You’ll see the BadVista website in the first page. If you search for “DRM” you’ll also see the DefectiveByDesign web page as well. This is big, it means that lots and lots of websites are pointing to those websites when they’re talking about those subjects!

If you care enough and you have a Google user and is using the personalised Google search, you could search for those terms and press the up arrow symbol on those sites to make them go even higher in the rank. Can we make both be the first? I did my part already.

…and our friend Mauro Persano won in two categories: 2nd on Intel graphics and 5th on best game on the go.

The game, Protozoa, is a retro Petri-dish style frenetic shooting-the-hell-out-of-the bacteria, virii and protozoa stuff that comes in your way. You can play with a PS2 (two-analogue sticks) control, one for the movements and other for the shooting, or just use the keyboard. The traditional timed-power-up and megalomaniac explosions raise even more the sense of nostalgia.

You can download the latest Windows version here but don’t worry, it also runs pretty fine with Wine.

Have fun!

Another fine example about how the open source community can be impressive, even when comparing with the biggest software companies.

Yesterday we had a gig at our annual Music Evening and I needed to edit the videos to upload them on my wife’s website. I go on cinelerra‘s website download page and get the Ubuntu 8.04 repository, update the package listing and try to install cinelerra:

sudo apt-get install cinelerra

It should be that easy but unfortunately the repository had an error:

Err http://repository.akirad.net akirad-hardy/main akiradnews 20080417
 500 Internal Server Error
Err http://repository.akirad.net akirad-hardy/main libguicast
1:2.1.0-1svn20080530akirad1
 500 Internal Server Error
Err http://repository.akirad.net akirad-hardy/main libmpeg3hv
1:2.1.0-1svn20080530akirad1
 500 Internal Server Error
Err http://repository.akirad.net akirad-hardy/main libquicktimehv
1:2.1.0-1svn20080530akirad1
 500 Internal Server Error

Well, with nothing else to do about it, I’ve followed the instructions on the website saying to email the guy that put the packages in place, which I did. Seriously, I thought it would take a while (days?) until the guy could have time to go home, do whatever he wanted to do at home, check his emails, talk to the ISP, bla bla bla.

To my surprise, after exactly 1 hour and 20 minutes he replied (in English and Italian) that the packages were reloaded, I should be able to get it, which I did, and indeed, worked absolutely fine. I now have my videos edited.

The “guy” was actually Paolo Rampino, which I thank him very much, but also I’d like to acknowledge once more the power of the open source community. I wonder if I had any much more serious problems (security) with a copy of Windows or Office if Microsoft would take 1:20 hours to not only answer, but to fix it!

Thanks again Paolo, you made another user very happy indeed.

I still code in my spare time and for a while I’ve been gathering some numerical methods I did at university in an easy-to-understand generic C++ package. Despite being easy to understand, I also tried to implement the best method I knew for each problem.

The root finding algorithm is based on Brent’s method which is, in turn, based on the secant method. If everything fails, it throws an exception and you can use the safe bisection method.

The integration is using the Romberg’s method, which is a further extrapolations of Simpson’s method which is already much better than the trapezoid basic method.

For Interpolation I’m using the Natural Cubic Spline but would like to implement other types of splines (like complete, periodical, clamped etc). The interpolation is working, but I couldn’t managed to get the coefficients right yet.

Other codes I have are Monte Carlo, Runge-Kutta and Markov Chain (this one using boost graph library for C++) and will be integrated soon. I’ll let you know when it’s done.

Last week I did two silly but still quite funny projects: word search on protein sequences and chat bot using markov chains.

Word search

Searching for similar sequences among the known proteins to understand evolutionary paths and function similarities is a powerful algorithm called BLAST. Following the same lines I spent a few minutes to develop a similar (but not quite the same) algorithm to search for all dictionary words on all UniProt sequences.

The algorithm will look into a dictionary (say /usr/share/dict/words) and for every sequence it’ll find the words in it, allowing a few (no more than 2) amino-acids to be between the letters. Than it gives a weight to the match based on the size of the word and the “added length” due to additional amino-acids in between. You can also restrict the number of letters on the word list to avoid too short or long words.

The results for 7 to 9 characters words in English searched only on Swiss-Prot entries are:

STRANGE  1.4
CHARADE  1.75
SLEDGED  1.75
MEALIER  1.75
DEMEANS  2.33333
DETAINS  2.33333

Markov bot

I knew a markov chain could have true wisdom but I never tried to make it actually talk to anyone until a friend told me about MegaHAL (much better than my markov model, of course). I decided to give it a try and got the XMPP bot from the Python examples and plugged in my markov program on all phrases spoken to it.

In the beginning it was awful, of course, it knew nothing. But with time (and hard work from some friends to actually teach him about leisure, programming, physics etc) it could babble a few coherent words once in a while.

The results are complete rubbish, below is a sample of the complete non-sense conversation I had with it now:

rengolin: how are you doing? have you managed to find the answer to Life, The Universe and Everything?
oraculoids: with an amazing company makes the basic points of some operator
rengolin: operator or not, you must come to reality and sense the danger there is in talking like that after such a complex and meaningful question
oraculoids: dead as an interface to follow an animal
rengolin: whatever animal you prefer, I would say but it seems that you are now driving the conversation
oraculoids:
rengolin: aha! I know you didn’t have answers to everything!
oraculoids: how is in between and pray
rengolin: I don’t pray too much, really, I’m not that kind of a person anyway…
oraculoids: but at the conversation it’t fall into them!
rengolin: I would never say that, especially not with you my friend
oraculoids: A design pattern is a practical laywer
rengolin: lawyers have no purpose other than screwing up the poor
oraculoids: hi bot
rengolin: I am not a bot
oraculoids: field of some of life, love them!

Quite stupid, I might say, but still I had a lot of fun doing it and that’s what matters… In spite of that, though, megaHAL is quite an impressive program and BLAST is a very powerful search mechanism.

Mike Stonebreaker (Ingres/Postgres) seems to be confused as well…

First he said Google’s Map/Reduce was “Missing most of the features that are routinely included in current DBMS”, but earlier he said to ditch RDBMS anyway because “modern use of computers renders many features of mainstream DBMS obsolete”.

So, what’s the catch? Should we still use RDBMS or not? Or should we still develop technologies based on relational databases while Mike develops himself the technology of the new era? Maybe that was the message anyway…

My opinion:

MapReduce is not a step backwards, there are sometimes when indexing is actually slower than brute-force. And I’m not saying that on insert time the indexes have to be updated and so on, I’m saying in the actual search for information, if the index is too complex (or too big) it might take more time to search through the index, compute the location of the data (which might be anywhere in a range of thousands of machines), retrieve the data and later on, sort, or search on the remaining fields.

MapReduce can effectively do everything in one step, while still in the machine and return less values per search (as opposed to primary key searches first) and therefore less data will be sent over the network and less time will be taken.

Of course, MapReduce (as any other brute-force methods) is hungry for resources. You need a very large cluster to make it really effective (1800 machines is enough ) but that’s a step forward something different from RDBMS. In the distributed world, RDBMS won’t work at all, something have to be done and Google just gave the first step.

Did we wait for warp-speed to land on the moon?! No, we got a flying foil crap and landed on it anyway.

Next steps? Many… we can continue with brute-force and do a MapReduce on the index and use the index to retrieve in an even larger cluster, or use automata to iteratively search and store smaller “views” somewhere else, or do statistical indexes (quantum indexes) and get the best result we can get instead of all results… The possibilities are endless…

Lets wait and see how it goes, but yelling DO IT than later DON’T is just useless…

UPDATE:

This is not a rant against Stonebreaker, I share his ideas about the relational model being far too outdated and the need for something new. What I don’t agree, though, is that MapReduce is a step backwards, maybe not even a step forward, probably sideways.

The whole point is that the relational model is the thesis and there are lots of antithesis, we just didn’t come up with the synthesis yet.

Long way ago, when I started using Linux, lots of people laughed at me: “What an absurd! You have to compile your own kernel, what do they want with that? They’ll get nowhere!”. Well, things have changed a bit in the last decade and Linux grew up as a very mature, modern and user-friendly operating system as we (not them) all expected.

OS companies didn’t believe at start but with time Linux became a nuisance, than a problem and now it’s real competition. Not only Linux (or rather GNU/Linux) but all free software and all the free licenses like GPL, FreeBSD, CC, etc. Linux is real business, it’s more stable, faster, better designed and change so much faster than any other OS in existence both for security patches and new features. Lots of companies today contribute to free software without charge or restrictions, just because we gave them so much without charge or restrictions (and it turns out as profit too!).

But last year something I wasn’t expecting happened… The biggest OS company for the last 15 years did a move so stupid that I couldn’t believe. Windows Vista was not an operating system, it was a joke, a *very bad joke* indeed. It reminded me the first upgrades of the first Linux distros back in 94, it was a nightmare.

Well, seems like the free software community learnt a lot about deployment, user interfaces, quality assurance, software development strategies. On the other hand, Microsoft seems a bit amateurish when trying to fix the previous mistakes. Every round it gets worse, I wonder where the good programmers they use to have are now…

Well, better for us, Ubuntu seems to be the new OS of choice for many previous Windows users and with recent Microsoft moves it may become more and more often… Luckily they’ll force everyone out of XP (the last minimally decent thing they did) as they did to Win2000 (the only reasonably decent thing they did) and people will migrate to Ubuntu instead of Vista… Let’s see the outcome by next year…

Welcome to the October 28, 2007 edition of information security carnival. We have frauds, ID thiefs, virus, spywares, privacy invasion and more.

articles

Marc and Angel presents 6 Digitally Traceable Tracks We Unconsciously Leave Behind | Marc and Angel posted at Marc and Angel, saying, “I have compiled a list of 6 digitally traceable tracks we unconsciously leave behind as we trek through our daily routines. I have also included a hypothetical example of how easy it can be to track someone down online by tracing their online affiliations and dabbling with the information that is found.”

Wenchypoo presents The Shocking Ease of Breaching Corporate Security posted at Mental Wastebasket, saying, “Written last year, but the info is (sadly) still relevant.”

Falando pelos Cotovelos presents Airport (In)Security posted at Falando pelos Cotovelos, saying, “Airports are a major concern nowadays.”

Doug Woodall presents Its Halloween! Spooks, Specters and Spyware! posted at The Spyware Biz Blog.

Wenchypoo presents Barking at a Hole in the Fence posted at Mental Wastebasket, saying, “Written last summer, but still relevant today.”

reviews

Renata Vincoletto presents Dangerous Files you Have to Avoid posted at systemcall dot org.

tips

Scott M presents How to Change the Root Password to Get Into a Linux Box posted at System Notes Org, saying, “Get Into a linux box when you don’t have the password. Requires physical access.”

Wenchypoo presents Credit and Identity in Shreds posted at Mental Wastebasket, saying, “A shredder isn’t enough!”

MT presents Safeguard yourself from internet frauds | MT Herald Dot Com posted at MT Herald Dot Com.

Wenchypoo presents No Rest from Identity Thieves–Even After Death posted at Wisdom From Wenchypoo’s Mental Wastebasket, saying, “I experienced this myself when helping my husband clean up his parent’s estate.”

Karl Sultana presents Keeping Children Safe From Online Sexual Victimization posted at NoAdware Blog.

Wenchypoo presents Wisdom From Wenchypoo’s Mental Wastebasket: Choice versus Privacy Invasion posted at Wisdom From Wenchypoo’s Mental Wastebasket, saying, “More to do with consumer information security than anything else.”

tools

Infosec presents Managing your Information Security Projects on line posted at Infosec.

information security carnival, blog carnival.

« Previous entries Next entries »