Finding repeating lines in a text file

Today I learned how to find all repeating lines in a text file. My goal was to find out all lines in a text which repeat more than once. A dirty, but efficient way is using sort(8) and uniq(8) combination (thanks to my boss for a pointer).

$ sort /var/log/messages | uniq -d

Do not try to pass a file directly to uniq(8) unless it is sorted. Uniq(8) can find only adjacent equal lines. If you think lines are randomly dispersed throughout the file, uniq will not return correct result. Be careful.

Горюшко.

А если бы в начале двадцатого века Российская Империя захватила мир, то вместо маршрутизатора Cisco был бы “ЛВС высокоскоростной переключатель Светоч–4506”
И не TCP/IP, а ГОСТ 1500124.37.

Коллизии назывались бы оказиями: шли два пакета по ЛВС и столкнулись, вот такая оказия приключилась.

А синий экран смерти назывался бы Горюшко.

И да, никаких реформ языка, всё должно быть с “ять” и тверднымъ знаком.

Срываю завсы альтернативной истории, рассказанной в суверенном.

IPv4 free address pool is empty. No reason to panic.

What Happened?

ICANN announced there are no more IPv4 addresses. What they said was IANA have delegated the last blocks to RIR’s. Does it mean you can no longer get a public block of addresses? No. You still can get a brand new block from your local RIR. Or yet unallocated block from your local ISP. Meaning, there is NO reason to panic.

How IP address allocation works?

For ones, who does not know how IP addresses are given out. The process is the following. The original holder of the whole free address space for both IPv4 and IPv6 is IANA. IANA registers and delegates IP address blocks to Regional Internet Registries (RIR’s). RIR’s alocate IP addresses and IP address blocks to end customers. If you want to get an IP address, you have to allocate a block by registering it in your local RIR. Upon successful registration the block is allocated to you and you can do whatever you want with it, including reselling it. Currently most IP blocks are allocated to ISPs and big corporations. End users are mostly getting addresses from their ISPs.

The allocation works as follows:

ICANN (IANA) -> RIR -> Organization (ISP, corporation)  -> End user

What really happened?

What really happened is IANA’s “IPv4 allocation department” is going to be renamed to “IPv6 allocation department”. They’ve done their job with IPv4 – they have delegated all of them to RIR’s, they announced it, they had a fierce party. These are the processes behind allocating the last block.

Currently whole free IPv4 address space (not so big – 16 000 000 managed by each of 5 RIR’s) is managed by RIR’s and ISP’s worldwide. Meaning, IPv4 addresses space is going to be really exhausted soon. Not yet, but it’s approaching. Still there is nothing to fear. Even if it fully exhausts, the only problem of yours is not getting a pretty 4-octet address. Internet will remain running. Moreover, IPv4 addresses are a subset of IPv6 address. Even if whole Internet except you migrates to IPv6, you’ll still be available on an Intenet.

The ICANN’s announcement shall be considered as a last warning and a last call for IPv6 migration.

Inventions of The Egypt Revolution: Emergency Internet Shutdown

The last, but not the least invention of the Egypt revolution is Country-wide Internet blackout. Local authorities cut the Internet shortly after they realized the way riots were coordinated. Local authorities used their influence to successfully convince 5 major ISP’s to shut Internet down. Technically speaking, all routes leading to Egypt were erased from the global routing table (Here is more details).

The simultaneous nature if the blackout means it was prepared before it’s happened, by every ISP separately. Consequentially, the same operation may be already prepared for any country around the globe. Hence, if you are about to start revolution, do not rely on the Internet unless it belongs to you, or unless ISP’s are on your side.

Another important conclusion about the event is telecommunications and media are closer related to a government than an army in some countries. For example in Egypt army took a side of rioting people, but ISP’s instead continued their Internet blockade. I can’t understand how it’s possible for few companies to be more loyal to the government than the entire Egypt army. What benefits does government bring to the companies? What level does their cooperation have? These questions are to be answered.

The effect of the blackout are to be calculated. Nobody knows what happens when you disconnect 80 M people. Nobody knows yet how effective it was. Nobody knows where and when it happen next. The event brings shock and raises many questions.

31 of January 2011, Friday was a day telecommunications raised in personal weapon effectiveness chart.

Inventions of The Egypt Revolution: Web-based Riots Coordination

I am continuing to share my impressions about the current revolution in Egypt.  Another invention of it – online riots coordination. It has been originated in Tunisia, where riots were coordinated via Twitter. In Egypt it went a step forward. As I understood from the public information sources, all kinds of social web services were involved into the information exchange.

Previously, proper coordination was available to an army and a governments only, but now with Web 2.0 it became available to everybody. It is a big step forward towards expansion of the ordinary people’s power, bringing more possibilities of delegating a power to people and cutting power from a government. More precisely, it’s a big possibility to make a big step towards democratic society.

Currently the importance of the Web is seen by the governments all over the world. Hence, I expect more control over the Web. The authorities are taking over the Web for a while already, and they’ll continue. Currently web became a subject of a real hazard to public authority, hence I expect raising degree of control over the Internet. I do not even know in which way yet.

Talking about web in this way, It is hard not to mention that Internet and Web particularly are greatest tools of providing freedom to people. Everybody has to protect the freedom of the Web as his own freedom.

Inventions of The Egypt Revolution: Leader-free Revolution.

First time in the history, the revolution was not inspired by a charismatic leader, but was a result of discussions on the web. Growing dissatisfaction of Egyptian people was published in their blogs and heavily discussed, until exploded. The decision to go to the streets was taken on-line on a Fecebook and heavily supported.

Another distinctive property of the Egypt revolution is a social class who moved the activities. The revolution was initially made not by a working class, but a middle class. Firstly, it’s unusual for well-satisfied middle class to rage against the existing social system. Secondly, the whole picture of a middle class having no leader inspiring army and a working class to make revolution looks much more wise and fair than a dictated working class throwing down existing government to set their authority-desiring leader.

It reveals a whole new hope for a brand new social culture, when countries are ruled by decentralized well-educated society instead of dictators. By the way, this is called Democracy. Feel the difference to current forms of social orders called “Democracies” (hello, US, privet Russia).

T1 logo creation

Artists draw their sketches, painters draw their pictures, modelers render their 3D, hackers program everything. I have coded my logo. I know the result is neither beautiful nor creative if you compare it to professional designer’s job. But it’s not stolen, it’s hand made and it is mine.

The decision to code the logo came after observing my drawing skills. No drawing skills were discovered. Despite, desire to create my own logo remained. I have chosen Processing for the task, I have had planned to try the Processing for a while already.

Designing the logo did not take much time. A bit of symmetry, a full rainbow of colors, “t1” in text. “T1” is acronym for tiamat, which is my nickname proudly.

I apologize for the picture's quality. It's been taken by fingerprint scanner.

How it worked? I have coded the logo’s appearance. After I’ve added processing built-in frame-to-image exporter and ran algorithm in a way, that all colors would fill the logo. I have used HSV for the logo’s color so I got 360 frames on output. All of them were imported to GIMP and combined to a single GIF image. I do not remember the formula of combining frames into GIF but it was a simple task.

There is a java applet with a showcase of the logo, there is code (1, 2) of the java applet, there is a GIF image. Yes, the code is terrible, ugly, slow. As any other “hello world” program. I am proud of it no matter what.

University. Secondary Degree Obtained.

My diploma is blue what means I’ve completed my education without honors and outstanding results. The logo on the diploma in my hands is different from the logo in the picture. My diploma has a double-headed eagle on the front, which is heraldic of Russian Federation. In the picture you can see the heraldic of Soviet Union. The picture shows outdated heraldic to symbolize that our education is outdated.

My secondary degree is “Information Systems Engineer”.

Этимология компьютерных названий

Почему-то некоторые англоязычные названия звучат настолько незнакомо, что я даже не подозреваю, что название имеет значение, а не является нарицательными. Значений всех нижеперечисленных слов я не знал, считая что у них нет лексического значения. Узнав, был удивлен.

Vista, например, значит перспектива, возможности.

Fedora – фетровая шляпа. Такая же, как и на лого компании Red Hat. Необзательно красная.

Java (язык) зял название от кофе, а не от острова.

Jabber – болтовня.

Другие названия, наоборот, прикидываются нарицательными, но ими не являются. Слов wordpress и facebook в английском языке нет.

Frame relay simplified