Technology

Technology — No Comments
31
Jul 10

Dumping thread stack trace in Java & .NET

Multi-threaded Java practitioners know about the indispensible ways to taking thread dumps to see a snapshot of what’s happening in the JVM, and resolve ‘hang’ issues. There are plethora of options, ranging from simple command line tools and utilities to nice GUI applications to writing some code in your application. A sampling of such options:

Stack trace in Java

Command Line

If the application is running as a console application, you can try one of these:

Sending a signal to the Java Virtual Machine

On UNIX platforms you can send a signal to a program by using the kill command. This is the quit signal, which is handled by the JVM. For example, on Solaris you can use the command kill -QUIT process_id, where process_id is the process number of your Java program.

Alternatively you can enter the key sequence <ctrl>\ in the window where the Java program was started. Sending this signal instructs a signal handler in the JVM, to recursively print out all the information on the threads and monitors inside the JVM.

To generate a stack trace on Windows, enter the key sequence <ctrl><break> in the window where the Java program is running, or click the Close button on the window.
(Excerpt from http://java.sun.com/developer/technicalArticles/Programming/Stacktrace/)

If the application is not running in a console, you can use the jstack tool that ships with J2EE since 1.5

jstack [ option ] pid
jstack [ option ] executable core
jstack [ option ] [server-id@]remote-hostname-or-IP

GUI Tools

Since the JVM provides APIs to hook in and get the thread state and other information, there are several tools in the market that allows you to see the state of the running application including the thread stack trace. One such freely available option is VisualVM. In fact it ships with JDK these days. Although VisualVM is pretty self explanatory, here are some instructions to get you started. http://download-llnw.oracle.com/javase/6/docs/technotes/guides/visualvm/threads.html

screenshot of timeline in Threads tab

You could also use jConsole for similar purpose, but I prefer VisualVM. Check this for more information: http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html#DeadlockDetection

Figure 11: Find Deadlocked Threads

Programmatically capturing the stack trace

from within an appilcation, you can call : Thread.getAllStackTraces(). But if you want to do the same from a different application, ThreadMXBean exposes the needed data, which can be retrieved by using the JMX interface.

Dumping thread Stack trace in .NET

.NET does not seem to have the variety of options like Java. The only reasonable option I have seen so far is the Managed Stack Explorer. This can be run from the command line as

mse /s /p <pid>

or as a GUI application:

image

Since it is an open source (MSPL) project, I took a peek into its source to check the API used. It depends on some sample code released by the CLR Team called mdbg. This in turn is a managed layer on the unmanaged CLR Debugging Services API.

From what I see there seems to be no direct way to get a stack trace in .NET, apart from either using the debugger API, or rolling your own on top of the StackTrace class. Check this question on StackOverflow.

Technology — No Comments
24
Jul 10

Beware of the clone

Why Clone

In this world of concurrent, multi-threaded programming, functional style of programming makes more sense. And one of the key tenets of functional programming is immutability. Even in OO languages, a few benefits of FP can be derived if the objects are made immutable.

Clone gone wrong

We follow a similar approach in our application. But this is not always possible, especially the mutable by default approach of Java. To overcome this, we pass around clones of instances, instead of the instance itself. This works in most cases, but we experienced a very subtle bug last week, where one thread was changing the values on a clone held by another thread, in spite of having no reference to it.

The cause

After lot of disbelief and head scratching, the reason was there, right in front. We were calling the instance.clone() to create a clone. The clone function defined in the Object class (more or less) takes each variable and makes a copy if it and assigns it to the variable of the new instance. This works great for stack variable, but the for the ones on the heap, only the reference is copied, in effect both the clones have reference to the same instance of the member variable. In other words, it does a shallow copy and you might actually need a deep copy sometimes.

Before Clone

After Clone

The effect

Thus any change made to MyClassInstance.OtherClass after MyClassClone was created also changes the MyClassClone.OtherClass, leading to confusing side effects.

The solution

Once the problem is understood, the solution is obvious. Just override the clone method and clone any reference classes in it.

Check http://stackoverflow.com/questions/2890340/question-about-cloning-in-java for more.

Technology — No Comments
11
Jul 10

Microsoft LifeCam VX-1000 on Ubuntu + Skype

Quick note to self, to save some searching next time.

Microsoft LifeCam VX-1000 does not quite work on Ubuntu and even worse with Skype.
To get video working on Skype, start it as :
env LD_PRELOAD=/usr/lib/libv4l/v4l1compat.so skype

And to get the microphone working, run this in a terminal
sudo rmmod gspca_sonixj
sudo modprobe gspca_sonixj

Technology — No Comments
5
Jul 10

Software development methodology

Recently I had a good discussion on twitter about the different methodology out there and their relevance. Although I agree with @ravi_mohan in a broader sense of not following blindly and not being a process fanatic or slave, I do not think he was quite right in dismissing all methodologies as fraud meant for idiots.

Since I found twitter a very limiting medium to discuss the nuances, I would rather blog about it.

What is a methodology?

As per wikipedia: Methodology may refer to nothing more than a simple set of methods or procedures, or it may refer to the rationale and the philosophical assumptions that underlie a particular study relative to the scientific method.

Methodologies we use every day.

We follow some methods or another all through the day, including when we are coding. We follow a set method for source control, bug tracking etc. taking these at a higher plain and we get a software development methodology.

We apply certain standard solutions to specific problem type, and then a few people saw these patterns and came up with design patterns. They work in most cases, evolved after years of refinement and are guidelines that can be used if you encounter similar design problem.

Similarly, few people saw some patterns in how we go about the process of developing software, saw what works and what does not and came up with Software Development Models.

Like design patterns, they are not to be used everywhere, and certainly not without a thought. In fact like several design patterns are actually counterproductive, like a Visitor pattern in dynamic languages. Similarly some development process recommendations are only to work around problems in workspace instead of actually addressing them. If you know the problems and have the ability and power to fix them, do it. You can then skip the part of the methodology that no longer makes sense.

Dismissing the collective wisdom of people who were able to see patterns in day to day work and were able to abstract and present it, is not right. Saying that the patterns are bad because the people who saw the patterns are not star programmers is, like saying, Harsha Bhogle is not a good commentator because he is not a star player.

Development methods in the wild (world of open source)

All great programmers follow a method. In fact open source projects with great leaders have the most articulated development process. Take for example the linux kernel development process. These processes are what work for that project but certainly you can see influences of well defined process. Who can deny that the 6 month Ubuntu development cycle is a perfect example of a time-boxed agile release cycle?

Conclusion

If you are working even is a small team, a process that is agreed and understood by all is useful. Preferably this needs to be written down, but if the process is part of the culture it is not needed. This has nothing to do with the skill level of the team members, as appropriately demonstrated by the teams working on the kernel. Whether you arrive at the process, by taking a ‘template’ and arriving at what works for you or seeing what is working and then refine and codify it, makes no difference.

Even if you are coding alone, you follow some process, even if it’s subconscious.

Technology — No Comments
25
Apr 10

Text processing options

The last two weeks, I have been researching options for processing free text. I think I have explored the entire spectrum of possibilities. Below are some notes that I can revisit in a few months and not spend the same effort again.

Background

I was looking at a way to process auto-generated tweets, like the ones on http://twitter.com/moneyvidya_com. Some sample tweets:

  • #moneyvidya arunthestocksguru (5 Star rated) says Buy Vijay Shanthi Builders – 6m (Monday 29 March 2010 @ 09:55 … http://bit.ly/bd5JgC
  • #moneyvidya arunthestocksguru (5 Star rated) says Buy Bhagwati Banquets & Hotels – 6m (Monday 29 March 2010 @ 09… http://bit.ly/9MzRDG
  • #moneyvidya NSV (5 Star rated) says Buy ACC – 6m (Wednesday 24 March 2010 @ 09:55 AM): http://bit.ly/b5xTrN
  • #moneyvidya justsurjit (5 Star rated) says Sell Sesa Goa – Intraday (Monday 22 March 2010 @ 10:31 AM): http://bit.ly/9lLo8U

As it is clear, the text follows a specific format, but has its own little variations. I intended to process the ‘insights’ and see each expert’s success rate. Although I never got around actually completing the task, I did learn a lot about text processing.

Approach

The apprentice – Regular Expressions

The first approach was the most obvious one – regular expressions. I am sure RegEx would have addressed the particular task at hand. But the parsing expression would become a convoluted mess very soon. So I started looking for better alternatives.

The strict teacher – Lexical Analysis

Lexical analysis starts where regular expression give up. This also needs pretty strict rules on the allowed input text, but the rules could be a lot more flexible and easy to comprehend.

I especially enjoyed using Irony, which makes it trivial to convert BNF formed rules to C# code. There is a good gentle introduction to lexical analysis using Irony on code project.

The guru – Natural Language Processing

Processing test using tools like NLTK, allows you to parse and understand any unstructured text and understand it. Although this gives you maximum freedom, it also needs a lot of work to get right. For this to produce good results, be sure you have lots of data to be able to tweak and test your implementation. I guess this is the reason Google and co., can do so much better at translation, since they have huge data available for improving.

Conclusions

I don’t have one :) . I guess, there are several ways to solve a problem, and half the solution is to identify the best way to solve the given problem. As for me, it was a good learning exercise and may come in handy if I ever write a DSL.

Technology — 5 Comments
11
Apr 10

Why I still use paper

and pencil for jotting ideas…

Visual Note-taking Conference Call Notes by Austin Kleon.

Try that electronically!

A quick list of advantages of the good old way of taking notes:

  • When ideas flow, only pencil can keep up with their speed.
  • Doodling actually helps with the idea flow.
  • Ideas are random, typing is linear.
  • You can get away with drawing something on paper to depict a long sentence. Doing the same on a computer will mean sidetracking into finding a clipart.
  • No pesky distractions, including those from the spelling or grammar checkers.
  • My best UI design always starts on paper
  • Easy to create mind maps.
  • Paper is truly portable, I can take it anywhere.
  • Only once I have all the thoughts on paper, do i try to organise them and make them electronic

How about you? What do you prefer, electronic or paper based note taking?

PS: I do like freemind when I am in mood for electronic note taking.

← Older Entries