deostroll

If you would have read my prev post, you'd notice that I've retrieved my twitter feeds in JSON format. My first inclination was to use XML because I've done some XML processing before. But that was way back on windows machines. It uses activex components to do the stuff. But activex is limited to windows operating systems & IE. What about other OSs?

Then I started thinking of XML processing on other browsers, and found out that FF has in-built capabilities to do XML processing. Then I thought, okay, most browsers would probably have some xml processing capability, but it is not something innate; the code I am going to be writing in javascript (for e.g.) for one browser will not work in another! Or else I am going to have to write separate code, one for each browser. That's hectic. But hopefully these problems will get alleviated in the upcoming version of javascript. That will take a long time. Who knows I'd probably end up looking like Gandalf by then.

So XML was out. I was forced to look at other formats and I came across JSON. It stands for JavaScript Object Notation. It is almost a text-like way to define objects in javascript. I first came across JSON when I was trying to work things out with Moxicode's wysiwyg opensource editor called tinyMCE. It uses JSON extensively. But I hardly realised this is a good data-exchange format. But I think it is limited to browsers for now. If you visit www.json.org they'd probably tell you it is adaptable to any programming language regardless of platform, and I am aware of json parsers out there, but I personally feel programmers are not that keen on json as a medium for data exchange over the internet. But as far as javascript is concerned JSON rocks!!!

Now that I understood that JSON was the way to do it, my next resolve was to retrieve it via HTTP. Normally we'd use XMLHttpRequest objects for this process. You'd have to use the activex equivalents on IE (I guess it is MSXML2.XMLHttp). But when I got the code executing I got access denied errors on IE, and on FF I got an access restriction because of some URI violation on FF. This had something to do with cross-site scripting (and same-origin policy). Such things can actually compromize your security by leaking critical information off to other sites for e.g. from your browser. The people at www.json.org mentions about this flaw. Sometimes it is required to have some cross-site scripting functionality. But such issues were never thought of initially when the XMLHttpRequest object guidelines were being designed. Christ! When you think of interoperability you always end up on top of somebody's fence!

But I guess the cross-site scripting issue is being taken care of in the next release of the w3c recommendations. I had to finally go with twitter's way on this one. Now I am aware you can "inject" some dynamic javascript via server side programming. Thats exactly what this does:

<script type="text/javascript" src="http://twitter.com/statuses/public_timeline.json?callback=loadTweets"></script>

The JSON will appear in the javascript in a format similar to the following:

loadTweets(<JSON output>);

All you have to do is to include another javascript file prior to the one above which contains the definition of loadTweets(). Its that simple.

Here is a way you can add your tweets to your blog.

Things you need & basics

First of all you need a twitter account. (http://twitter.com). If you don't know what twitter is take time to watch the video. The cool thing about twitter: you can talk to its servers from virtually anywhere via the internet and get your tweets out, (as well as in). This is accomplished via the twitter api, which is nothing but a series of RESTful interfaces. We only need the api to get our tweets out. But for this you must set your feeds to appear on the public timeline.

Secondly, your blog! I've used my blog @ blogger.com for the exercise. But I assume you can do this from any blog site where you have a lot of customizable options - one of them being the ability to add custom HTML/Javascript.

Thirdly, a place on the internet to store your javascript (so that it is globally accessible). I've used geocities, but I think I'd have to move because they've decided to close down somewhere in the future!

Diving in

I am going to assume you ready to enter some custom html & javascript on your blog page. Just copy and paste this html/javascript:
Basically there are two div tags - one with id loading, and other tweets. The 1st one will appear whenever I an trying to fetch my tweets, and the 2nd when the tweets are fetched. So its a kind of ajax-like behaviour going on here, but not quite ajax. Your feeds will appear in the unordered list (with id = twitter_update_list). But if you know html & javascript you can put anything you are comfortable with.

Coming to the javascript part. Almost 75% is I guess twitter code itself. I've used it for convenience, because in that script any urls (which would normally appear as text in your feeds) will get converted to hyperlinks. (Plus, I was lazy to write my own way of formatting my feeds).

Now getting to how-to-get-the-feeds part. That is where the 2nd script comes in. I've conveniently stored it on my personal website on geocities.


Notice the url I use to get my tweets:

http://twitter.com/statuses/user_timeline/deostroll.json?count=1&callback=loadTweet

This is the request I've to send over to the twitter servers so that I can have access to my feeds. Here I've asked for just one feed. You can put as many as you like but there is a limit set by the api standard. The callback things is the javascript function that will get executed once the script loads. Note that deostroll is my username in twitter. You'd have to replace this with your username. (Thank god I've such an unique name)! Most of the lines of script above are commented. Hope they'd give you a basic picture.

Also note .json part (in deostroll.json). I am explicitly telling the twitter servers to output my feeds in JSON format. This is more compatible with javascript. You have the option to retireve your feeds in RSS, XML, & JSON.

So there you go, Happy programming.

PS: I've found out the hard way that you can't actually use the XMLHttpRequest object to send http requests to any server in general. It is a security restriction in most browsers. That was my original approach.

It is hard to explain why a hacker has the urge to hack. And in a small way I feel that same urge too, I am not that deep into hacking. You should not think of the anti-social meaning of the word 'hacker'; hacking has its pros and cons too!

Lately I've been drowning swimming through the "clouds" of cloud computing. There is a luring promise that any web application is possible via the cloud. If you are still wondering what the term means, may be you should read this.

There are a lot of cloud computing servers around. Google's AppEngine, Microsoft's Azure Services Platform, etc are few examples I can think of now. You can probably go there and download their sdk, and get started with building your own web applications now.

Before you start to swim in it, I'd rather point out, most web applications today, whether cloud or not, require some database engines running behind the scenes to store data and do other stuff. The whole website would run based on the data stored in it. Simply static web pages won't do these days. When you think of the cloud, this is a challenge for any cloud engineer. They are finding ways how to enable cloud based database support. It is not that most clouds don't have anything at the moment; they are inferior compared to conventional database engines.

Google has something called GQL. Its not so hype. It has a learning curve associated with it, and it does not support most of the traditional things a popular database engine does.

I first learned about google's cloud platform more than 3-4 months ago I think. You have to write your applications in python. I first learned about GQL about that time too. Even the GQL is somewhat python based. (The querying is in SQL). Lately I've heard that they've also extended support for running Java programs in the AppEngine server; now you can write your web application in java.

So its now time to graduate to java! More specifically I am driven by another motive. And from here it can either take a java route or a python route. Either way I'd be making use of the java runtimes somehow. To make more sense of what I've just said: I know database support for the appengine is limited. I don't find GQL to be the best database engine. So I was thinking...so GQL basically sucks. If not GQL I'd need to persist data and do some querying on them. You know it seems like I've done this before somehow. I used to open MS Access, create tables, store/query data, create reports, etc. I've even used to do the same things via a vb application too. But on a windows platform if you need to "talk" to ms access files, you need the MS Jet Database Engine. I guess you can talk to this engine via ODBC, OleDB, etc. Now how do I get this engine uploaded to the appserver? What guarantee is there that this would work? I guess none!

This was where the news that they've extended support for java came to the rescue. I am curious to know the os Google would be using to host its AppEngine applications. And I don't think it is impossible to find out either. (I just need to get my python skills cooking). But I'd bet it would most probably be a unix/linux platform. And they'd probably be using apache to do their hosting too. This would mean that the jet engine would not work! Therefore ms access is out! The alternative to ms access is OpenOffice Base. AND IT UTILIZES THE JAVA RUNTIMES.

My ultimate motive is to make a simple discussion forum web application. And I don't mean to do some serious relational database stuff here. So I guess this will do.

Recently I was engaged with the task of developing a system whose main purpose was tracking an activity. The goal of the system however is something more huge – to automate processes within my department. I do have my doubts on the usage of the word ‘automate’. I think it suggests to somehow carry out the process without manual intervention. However I realize that we cannot fully automate all the processes. Well, ultimately the system documentation is up to me, so I reserve the power to change anything… (*feeling a bit of irony here*)

As with all the projects over at my work place people expect to see this product rolled out over night. However I was quite able to convince them that this is something that has to be long term. I was able to understand that they expected a “lot” out of the system.
We already have a system for activity tracking but it is purely what it is called. We manually enter all data to the system. There are reports generated based on this data. However we are not able to arrive at a statistic that says ‘this is the service level of the department’, or ‘this is the service level of this person’. This means we need to take an inventory of all activities and fix SLAs for each.

SLA-ing is a whole process in itself. It involves dialog between two parties, mainly between one who provides a service, and one who vies that service. We are the service providers, and the people in the Operations department vie for our services. Ideally we are supposed to reach to an agreement with them. That process involves explaining to them about the service. What is the expected level of service the technology (used in the process) provides, what inputs the service requires, etc. it is a laborious process. And it depends on their exposure to the technology.

We do not have any operating procedure to ensure that their exposure levels are all the same. I believe it was never conceived before. Currently the education is all word-of-mouth. What we can do to ensure a consistent awareness is conduct ‘awareness appraisals’ and ‘assign’ someone a level of understanding, a grade. Although we can work without such a requirement, we need one to ensure our system becomes accepted, and people understand the work we do. This is something we have to think about so I’ll stop short here.

So for the time being we are fixing the SLAs. We know this is something that needs to be revised frequently until processes stabilize. People who have to work constantly with this system are kind of worried about the implications. They are worried about how the management would behave if such a system was in place. Actually everyone “wants” a system in place; they all welcome the initiative, but some are too worried about the way they are going to work in the future. Perhaps a manager’s role is wanted here; he/she needs to break the ice, and make people understand the need for such a system. This might require the manager to also come to an agreement with the operative users of the system. But I’m not getting into that.

Currently we are going activity by activity. People expect the input to come from the system; they expect the output to go through the system. It should get delivered to the appropriate clients. Can you imagine the hell I would have created if I came out with an implementation over night? Even my understanding of all the activities are low. I need to know each and every activity in concise detail. I have to know what are the inputs for an activity, what process is done, does this process go through an authorization, what output it should give, and in addition to this, what related activities to be carried out. It is a great deal of struggle to actually realize the last point. Some of these ‘related activities’ have to be carried out to ensure an industry standard is in practiced. People don’t often realize this, and they don’t perform the activity. I understand that the reason I am struggling with the last point is because of an incomplete adherence with the current standards.

This is where I thought the goal of the system should be modified a bit. While the system is being implemented we must ensure an activity is completed as per norms in an industry standard. So currently I work with my team mates and understand all activities – the way they do it. I come out with process diagrams as per the current standards requirement. So everyday it is interacting with a lot of people, and the company standard practice. And everyday I find loop holes in both.

Dealing with the loop holes with the standards is the non-frequent issue. The clauses (in the current standard) are vague; however they deal with all activities. We currently only have one for support and development. We do not have one for reporting, or voice file editing, which are daily activities happening within our department. I expect to see consistency from a standards perspective. If I don’t see it I begin to doubt the standard. Sometimes we are left to our own interpretations. That’s when the problem occurs.

Here I have to rely on my managers. And I here I realize if I become a manager I need to be educated about the standards. Why? Because in case of doubt that the only way.

So far this is all what I’ve done. I know this might take forever. And I wish there is someone to answer my questions along the way. So far it has been interesting. Actually everything I am doing sort of relates to a book am currently reading; Just Enough Structured Analysis. It is a book about system analysis. A must read for any system analyst.