Archive

Archive for the ‘Software’ Category

Renaming ntop to entop

2010/08/16 1 comment

So the ntop application I released yesterday has received pretty nice feedback from friends and unknowns but apparently there is another ntop application out there; the “other” ntop is “Network Top” and is found at www.ntop.org. So I’m changing the name of my ntop to entop! It is a corny name… I know… but I liked the name ntop so this will just have to work + google doesn’t show anything software related when searching for entop (except for some Finish stuff which doesn’t look software related :D) so I just picked that name.

So… ntop is now entop which stands for “Erlang Node top”. Enjoy it here:

http://github.com/mazenharake/entop

Also; here is a new screen shot! It looks exactly the same as the previous one… but says entop 😛

Categories: cecho, entop, Erlang, ntop, Software Tags: , , ,

Announcing ntop – A top-like monitoring tool for Erlang nodes

2010/08/15 8 comments

The name in this post is old; the application is now named ‘entop’, just to clear that confusion.

Introducing ntop 0.0.1

ntop is a tool which aims to be similar to the unix tool ‘top’ but instead of displaying the OS processes it displays the processes (and various information) of a given Erlang node. If you don’t know what ‘top’ is then see this wikipedia page.

ntop uses cecho (must be version 0.3.0 or later)

I wasn’t too sure of what information one would want other then what I put in it; I have only my own and my collegues’ experience in what we need when monitoring systems so to make sure that this can fit anyone I made sure to make the columns and headers customizable enough to print out different information (which might be more relevant to other people). I’ll go through how to write a different version and how to extend ntop in a different post.

ntop is released under the 2-clause BSD license, do what ever you want with it. Here is a screenshot on how it looks like:

If you try it out then please let me know if you find it useful and if there is something missing or needed! I’m sure there are bugs as well and I’ll fix them as I go.

You can find it (and cecho) on my github page: http://github.com/mazenharake

Enjoy.

[Edit]: Doh! Forgot to give the link to my github page. 🙂

Categories: cecho, Erlang, ntop, Software Tags: , ,

Euler 11

2010/07/08 1 comment

After packing yesterday I thought I’d spend the last half an hour to solve the Euler 11 problem. I had used pen and paper in the car on the way home (I don’t drive myself 😉 to come up with a nice enough algorithm. However no matter how I twist and turn the problem I realized that I still need to check every possible solution. The first naive way on paper was to check every direction.

This is an example; assume we have a (zero indexed) 10×10 grid. and that our position is say at 4,4 (in blue) then the initial idea was to check all possible directions (below in red)


x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x

I soon realized that the opposite of each direction is just a reiteration of something that was already calculated. E.g. 4,4 -> 7,1 is the same as 7,1 -> 4,4 . This means that it is enough to just test half of these.


x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x
x x x x x x x x x x

Now I only have to make sure I don’t get out of bounds. Since I’m traversing the grid from top right to bottom left i want to do as few checks as possible. First thing is that 3 out of 4 checks are made on the left side of the position, this means that I need to check that the position is x >= 3 and y > 3 and y < height – 3.  The last one only needs to be checked when the position is y >= 3 only. After that I simply take the max of those values.

This yields the following in python 3:

def local_max(y, x):
    llarge = drdlarge = ularge = dlularge = 0
    if x <= 16:
        # Check from position to right
        llarge = grid[y][x]*grid[y][x+1]*grid[y][x+2]*grid[y][x+3]
        if y >= 3:
            # Check from position diagonally right up
            dlularge = grid[y][x]*grid[y-1][x+1]*grid[y-2][x+2]*grid[y-3][x+3]
        if y <= 16:
            # Check from position diagonally right down
            drdlarge = grid[y][x]*grid[y+1][x+1]*grid[y+2][x+2]*grid[y+3][x+3]
    if y >= 3:
        # Check from position up
        ularge = grid[y][x]*grid[y-1][x]*grid[y-2][x]*grid[y-3][x]
    tmax = max(llarge, drdlarge, ularge, dlularge)
    # print("Local max:", tmax)
    return tmax

Then it is just a matter of iterating across the grid and get the largest value:

def traverse_grid():
    largest = 0
    for y in range(0, 20):
        for x in range(0, 20):
            candidate = local_max(y, x)
            if candidate > largest:
                largest = candidate
    return largest

if __name__ == '__main__':
    start = time.time()
    print("Answer:", traverse_grid())
    print("Completed in", str(time.time()-start), "seconds")

Two important lessons followed (This is the biggest reason to learn a language by actually USING it):

  1. On line 12 in the local_max() function I put x >= 3 by mistake. I thought everything was ok when I ran the code because I did get the right answer. However after look at the code when writing this blog entry (I wrote most of this yesterday evening and revised it today) I noticed that x >= is wrong! Python has a “feature” which is pretty dangerous if you don’t know about it. If you have a tuple e.g. grid = ((1,2,3),(4,5,6)) and then do grid[-1][0] the result is not a crash, it is simply 4. This is because in python you can slice a tuple as well! Very important detail!
  2. Because I usually program Erlang I sometimes put a comma at the end of a line by mistake. On line 5 in the traverse_grid() function I put a comma by mistake after the function call. This is another “feature” in python which means that I’m creating a one-value tuple. I was banging my head senseless trying to understand this late at night and simply extracted the first element but I couldn’t understand why. Today I asked my friend to be my Rubber duck and while explaining the code I realized what I was doing. This is also important to know because it “fails” silently if you don’t realize what is going on as an Erlang programmer

That’s all…

I’m off to Turkey! See you at the end of the month!

Categories: Python, Software Tags:

Eirc – An IRC client library for Erlang

I just released eirc. An IRC client lib. I haven’t done any proper testing and no performance testing either but it works for writing simpler IRCBots which was my intention.

Next step will probably be a git commit watcher/announcer which uses eirc to announce events from a git repo. I’ll figure out how to do that later.

Btw: Anyone who knows any guides on how to work with .git or bare repos (Read: How they are structured, which files that say what and how to extract various information from the binaries (if there are any)) then please let me know.

peace

EDIT:

The link to the library is here: http://github.com/mazenharake/eirc

Categories: eirc, Erlang, Software Tags: ,

Cecho 0.2.0

Just released it here: cecho-0.2.0

Now that it supports windows and proper key-input then maybe I’ll use it for something slightly more useful then my examples found inside 🙂

Categories: cecho, Erlang, Software Tags: ,

cecho – An ncurses library for Erlang

2010/02/16 3 comments

I’ve been thinking about monitoring tools lately and have been thinking about how to get nice UI’s through a terminal window. I had a project going on with the wxErlang library but I scrapped it (or rather put it aside for a little while) because the reality is that most system don’t have a window manager and you usually connect to a machine through ssh.

So I created cecho.

cecho is an ncurses library (wrapper). It works nicely on my linux machine which is as far as I have tested it so it is really pre-alfa but it seems to work. It can be found here: http://www.github.com/mazenharake/cecho

If you download it then check out the cecho_example.erl file for some simple examples on how to use it. It is intended to stay as close as possible to the ncurses API but in some cases it is impossible so I try to make it as alike as possible.

Here is a hello world example:

hello_world() ->
  application:start(cecho),
  cecho:cbreak(),
  cecho:noecho(),
  cecho:move(10, 10),
  cecho:addstr("Hello world!"),
  timer:sleep(2000),
  application:stop(cecho).

Hopefully this will become useful for someone and some cool stuff come out of it.

Important: This library does NOT support keyboard input yet… I haven’t figured out a proper way to implement it without it being a hack so therefore it will have to wait.

Important2: For some reason the Erlang VM crashes when one do a cecho:move to the max coordinates (the values of cecho:getmaxyx -1) and then add a character (or string). I don’t know why but I think it has to do with newline moving the cursor out of bounds or something… I’m still looking into it.

Have fun… I know I will 🙂

Peace out.

Categories: Erlang, Software Tags: ,

Bridging SVN to Git

At work I have a problem. The problem is that I am doing many things at the same time… Now usually this is not a problem but in this case it is. The reason why this is a problem is because I’m the developer and maintainer of a few components that are in both production (I need to bugfix etc), development (I need to add enhancements etc) and research (trying out design and concept) and I need to maintain several versions of the component in each of these. But the core problem is that I don’t have the right tools to do this effectively. Since we are using SVN as a version control system I’m somewhat limited in how I can work.

To make it a bit more annoying I also have 2 laptops that I work on. One which is my personal one which is more customized, have the applications I want and I can install what ever I want on. The other one is my work laptop which must have certain applications, I can not add what ever I want and a few times a day automatically updates my Windows OS and reboots. My work laptop is honestly a pain in the ass sometimes.

Anyway. There are three things I would really like to achieve which I can’t when using only SVN:

  1. I want to be able to commit my work without having access to a central server
  2. I want to be able to create branches and merge between them without keeping track of when they were made or last merged and also I want to create an arbitrary number of branches to test stuff offline
  3. I want to be able to easily transfer my work from one laptop to another without going through the central server including a merge when I do so. E.g. If I made a small change on my personal laptop and then switched to my work laptop and did more work without the change make in my personal laptop and then want to change that work back to my personal laptop I don’t want to overwrite what I did there previously, I want to merge it!
  4. I want someone else to be able to contribute to the component so that if someone else noticed something while I was away and fixed it then they should be able to allow me to pull it in without sending me an email saying here is the diff (bleeeh!) without me having to update from the svn repo and get conflicts

That’s what I want(!) and fortunately I can achieve all this with Git (the rest that comes with it is just extra!). I should mention that there is something called git-svn but Since I don’t want to work with a whole repo, just a part of it, I didn’t want something as heavy as git-svn. Git-svn supports almost everything and tries to “merge” the git and svn commands which is not what I want… I find it confusing and unnecessary.

Now the problem I had to solve is how do I create a bridge where I can have my own structure with a central repo for things that are ready to be commited to SVN? So after a few minutes I realized that his is what I want:

So this is a step-by-step guide to how I achieved this; maybe someone out there will find it useful.

1. Check out your directory from the SVN repo
Check out the directory you want to bridge. SVN creates a one “.svn” directory in each subdirectory that is part of the SVN tree (including the top directory). This directory is changed on every action that you do in that directory (and every parent directory will know about it). Git doesn’t do that, it sees things differently (fortunately! you will understand why later), git has only 1 directory which it will create. Anyway so check out the directory you want to track. Below is an example:

:~> svn co svn://hostname/repo/trunk/app
A     app/file1
A     app/file2
...
A     app/fileX
Checked out revision 2823.
:~>

2. Initiate a Git repo inside the SVN repo
Ok, so after we have checked out the SVN directory we need to initialise the Git repo inside the directory that was created by SVN.

:~> git init
Initialized empty Git repository in .git/
:~>

3. Have SVN ignore the .git directory
Since we don’t want to track the .git directory in SVN we have to let SVN know that we should ignore it. This is done by putting a propset on the top level directory (where the .git directory is). Here is how it is done:

:~> svn propset svn:ignore ".git" .
property 'svn:ignore' set on '.'
:~>

Node the last character in the command (the period), it sets the property on the current directory.

4. Have Git ignore the .svn directories
When we push and pull back and forth we don’t want git to care about .svn directories because they change every time any of the SVN tracked files change and we want to be able to push/pull to the staging directory without git getting confused. This is how it is done in Git:

:~> echo -e ".svn\n*/.svn/*" >> .git/info/exclude
:~>

A few things to note; the first .svn is to ignore the top-level .svn directory. The “*/.svn/*” pattern is to ignore all the rest of the .svn subdirectories. Here you can add more patterns; as an example I put the string “.svn\n*/.svn/*\n*.beam\n*.app” to ignore Erlang binaries and the Erlang application file (which my Makefile generates). If you are using other languages you can just adjust to whatever you want to ignore. Personally I’m only interested in tracking source files

5. Add the files you want to make available to Git
Now all your files are tracked by SVN and SVN is also ignoring git. Git however is not tracking anything. So do this by explicitly adding all the files you want to have available in the Git “cloud”, or if your exclude is specific enough you can do like this example:

:~> git add ./*
:~> git commit -m "Initial commit"
10 files changed ...
...
create mode 100644 file10
:~>

That’s it. Now you can pull this directory from where ever you want and commit to the stage directory until you are ready and can pull/push between computers. Have fun! 🙂

!WARNING!
There is (what I consider a really serious) bug in Git. This is how it happens:

  1. Create two directories E.g. foo/ and bar/ and initiate a git repo in both
  2. Create a file “baz” in both directories E.g. ‘echo “data” > foo/baz’ and ‘touch bar/baz’
  3. Go to foo/ and add the file to the git repo and commit it. ‘git add baz; git commit -m “whatever” ‘
  4. Go to bar/ and do a pull; ‘git pull ../foo/’ Git will silently overwrite the file in bar/

This behaviour is utterly stupid; git should warn of existing files or, even better, try to merge the files! I haven’t found where to submit a bug report yet but I will as soon as I find out where.

Hope this is useful 🙂 Good luck!

Categories: Philosophy, Software