Digging out a post from my previous blog. Thought this post should be seen by a global audience to see how some Big Indian Service companies deal with employees or potential-to-be-employees. This incident changed my outlook towards a ‘company’ in general. And after this incident I never kept the same respect towards any other HR policy.

The funniest aspect of the whole story is that, the HR person who was involved in this whole incident (which happened in the 2006-2007 time frame) is still with the same company (infact promoted as a Executive HR :O) and she completed her MBA education from the most renowened management institution (famous for its HR education) in India :O [I want to question the basis for her selection to this institution and the basis for her promotion. A person who does not have courtesy, decency and least of all a sense that she is playing with other's careers is selected to a premier institute and is promoted :O, The world is definitely not fair. I will definitely reveal the name of the company in this blog one day, not right now.]

– The post –

What I write here is about an encounter with the HR/Recruitment team from an Indian IT gaint, which is one of the top Indian Service companies. I am NOT going to give the company’s name here… as I am NOT doing any witch hunting here.

To start with, after my wife resigned from the company (I wrote about this in another post of mine), she started applying for bigger ones and made it to this company. People at home were happy that my wife had the calibre to make it big. All this happened in Nov 2006 and she was asked a question in the interview whether it is ok for her, if they post her wherever the company wants. To that she replied, no my marriage is in Feb 2007 and I cannot, so please consider putting me in Hyderabad. To that the person replied, That’s good, you did a good thing by saying this in the beginning itself. She came back home, but was still inconvienient about the fact that she might be placed elsewhere and wrote a mail to the HR here in Hyderabad, looking after recruitment, to consider putting her in Hyderabad.

Two days later my wife received a mail from the company stating that she was being posted to training at Chennai. To this she wrote mail to the same HR at Hyd stating that, please consider my position and consider putting me at Hyderabad. To that there was a reply from the HR stating that “You can join at Hyd, but you will have to wait till Jan”. Then my wife waited till Feb and seeing no mails from them, wrote a mail to the same person asking her [the HR] what the status is. The reply was given by another person who we assumed to be the junior to the HR (we initially sent mails to). He replied by calling us that you will have to wait for another month for the training to commence.

We waited till March, exactly a month, and wrote a mail to him as well as the first HR asking them when it would start. There was no message from their side and my wife went to meet them at their office in Hyderabad. My wife met that person who called her and asked him why is this getting delayed and he said, please bear with us it will take another month.

Like this the replies like “Next” month kept on going and finally we reached September 2007. When my wife called the first HR in Aug 2007, She [The HR] replied saying that “Everyone has valid reasons, we cannot please everyone. You might be offered a training date at Chennai in Dec 2007. Please Wait.”. We knew that applying to another company and joining that company would take at least one month. And since we already have an offer at hand from a good company, why take pains like that. We asked a person whom we knew in that company to know why this is getting delayed like this and asked him to see if he can recommend to the HR team to see to it that she gets a place at Hyderabad.

The person got back to us with a reply saying, the name of the candidate does not appear in the list of people to be offered a job at our company. The reason stated was she did not join on the first DOJ for which she received a mail in Nov 2006.

We were totally shocked and utterly furious on the HR team for telling us to wait eternally for a long time to just get back a news like this. We wrote a mail to the HR people asking them what the actual status is, as we got a message from our contact in the same company stating something different from what they had been saying all the while. To that the HR team replied in mail stating that “As we have already discussed, the offer stands terminated”. We in reply wrote a mail to the HR team as well as CCing their Hyderabad operations head and the Senior HR management asking them why the HR team of a big company like this is so incompetent, and what we should be telling about this company to other people who ask us about it. That HR team does not respond truthfully in e-mails or they keep you waiting for eternally long time.

We got a call from the HR team asking us to come and meet them on Wednesday [19th September 2007] the next, with the offer letter that they gave her. I was telling my wife not to go first and ask them what the agenda of that meeting would be. To that my father-in-law and my dad said, no probs we will meet and see what the out-come would be. I did not reply and waited for this meeting to end. And this is what happened on that day.

HR team was utterly adamant and they told my wife, “We have already sent you a mail regarding your NEW joining date and you did not respond, and hence the offer was terminated.”. My wife said, “I did not receive any”… and “Please show me any mails that you would have sent me.”. The lady [HR] was furious and replied “We have sent them, and we need not justify ourselves to anyone. We have them with ourselves”. To this my wife argued for some time and said “You have made me wait for ten months and now you are still being very unjust to me. Please tell me what I should be telling in any interview as to why I wasted ten months”… and to that the lady[HR] said “Who asked you to wait? You should have INFERRED from our statements about ‘next’ month that you don’t have a place here” and to this my wife got frustrated and said “You are lying to me, by saying that you have sent a mail regarding the new joining date. How can you be like this?”. To that the lady[HR] replied, “How can you blame us like this? This suggests that your attitude is improper and we don’t want such people at our company.“.

My wife got totally vexed with the HRs and got out saying that, till now I had a good impression about your company, Now I have lost all that. My dad accompanied my wife that day to the company and when she came out and told him what happened inside. My dad approached the HRs and said, “See, We understand that some communication gap has occured somewhere and we know mistakes happen everywhere. We are not questioning that, but you should understand that you have spoilt ten months of a candidate, please rethink about this and do justice here for whatever mistakes have happened.”.

My dad was told by the HR team, “We don’t talk to relatives of a candidate. Whatever we had to tell we have told the candidate. We don’t want to discuss anything more.”. To that my dad said, “Look, we are people who have seen bigger companies, so please don’t tell us as if we haven’t seen how to deal with such things.”. To that the reply from HR team was, “We don’t want to compare ourselves with others… We have our own policies and we work according to them… So what do you want finally? ‘An apology mail for whatever has happened till now?’“. My dad said, “Ok please lead me to your Senior HR manager and I will talk to him.“… and the reply from the HR team was… “Sorry, you cannot meet him, he is very busy.“.

My dad is as such a very busy man, and these HR people wasted half a day of his and they give such reckless and immature answers. After coming home my wife felt very sad for how they attacked her for the mistakes that they [HRs] made. My dad was telling us how immature the HRs were.

Initially we contemplated a legal move against the company for damaging my wife’s career for ten months. But, my wife was against it, as she felt it would make her look bad, as it might give a wrong impression to other companies that she creates problems everywhere. So, finally my dad has decided to write a mail to their senior management about how great their Team is and has decided to give them a piece of his mind.

Pratap

My old blog on Yahoo360 used to register at least 20 hits a week . Considering the random stuff I used to blog about it is difficult to say who used to read my posts. But after moving to WordPress my hit count was around 0 to 1 hits per month on average for the last one year. Now it has at least picked up a bit and is showing some 1 to 2 hits per week :O… Some improvement at least. But would like to get it back to my original hit count of at least 20 counts a week. I am very serious on hitting that number again and will try to blog more often this time . Will concentrate mostly on Technical stuff as I believe it will generate more hits (interest ) compared to writing random stuff, i.e. about movies I saw and my other comments.

Pratap.

PS: Waiting for some comments from people telling me whether this ‘vision’ for generating more hits is a correct one or not .

This document took me a long time to write :).  Actually had written it for my team and shared it with them some six months ago. Thought it might be a good idea to share it with everyone. Most of the stuff I have written in this document came with experience. All the other stuff required some painful browsing and learning . There might be a few spelling and grammatical errors in the document, but I assume they will not hamper the readability of the document (a bit lazy to correct the doc )

So, if you like the pdf pls drop a comment. If you think anything is incorrect in the document then please feel free to point it out and ‘suggest‘ your correction (this is a very important feedback you can give ).

SQL Coding Guidelines (Pdf) – The Link

 

Pratap.

PS: Will follow it up with a document on design guidelines. Think it needs some cleaning before I can upload it .

BTW. Sorry for disappointing you Girish … Another post on SQL Server… Will try to write something non-technical in my next post.

I was going through the design of one of our OLAP databases and in that database I found certain fact tables with columns which seemed to be fitting the description of a dimension rather than that of a fact/measure. I had long arguments with the designer who designed this database and still couldn’t get a convincing answer. He seemed to have started designing the database assuming this to be a fact whereas now he was confused between terming that field as a fact or a dimension. So, I did some reading on this aspect and found this article describing something called as ‘Degenerate Dimensions’. It explained in detail why some columns are more like dimensions and how they need to be modeled.

As such this discussion happened between us some months ago… but I started loosing track of the URL… hence I thought it might be a good idea to track it as part of a blog post . Also, it might be of some use to people who are designing OLAP applications.

Pratap.

   After my friend commented on one of my previous posts about O/RM and its relative advantages over traditional DB programming, I decided to take a deeper look. Over the past one week I have talked to some of my friends and colleagues who work (or worked) with databases and took their views on how good O/RM (NHibernate or Linq) is for database development. I went through one tutorial which taught me how to use Linq for my basic database needs. Overall I have seen a very divided opinion on using O/RM and now after going through one article by “Ted Neward” (link provided to me by one of my friends at Microsoft) have come to a better conclusion on what O/RM can provide and what problems it poses to the developers.

Some of the good points I could find about using a O/RM solution are as follows:

1. By using NHibernate or Linq we mostly create a mapping file which tells us (or the O/RM layer) how the tables map to different objects. This is enticing because my middle tier code (Java or C#.NET) is entirely isolated from the database implementation.
   1. i.e. I need not write stored procedures or DB-Functions etc at the DB layer and hence during development I can use SQL Server as my development platform where as during deployment I can deploy to a Oracle database.
   2. This is particularly advantageous when I have an application which I wish to sell to multiple vendors who have different database systems installed on their environments. By using an O/RM solution which helps us to cater to a wide variety of systems available we need not worry about which database system I wish to install my application on.
   3. This was the biggest advantage I found with O/RM.
2. I have seen other arguments on the web too…
   1. Writing stored procedures is similar to writing Machine code. O/RM is like a high level language, you can just use the object oriented paradigm and forget about writing SQL (which is like machine code) and hence you are following the latest trend . 
   2. A Developer needs to know only the object oriented paradigm and a language such as Java or C#.NET. Knowing other db access languages such as SQL can be ignored.
      1.  Somehow the above two arguments did not convince me. Especially after seeing the syntax of Linq it is obvious that SQL cannot be ignored. As far as I had seen in the tutorial I went through, Linq used the SQL systax wherever possible (at least for the SELECT queries).
      2. So, arguing that you need not learn SQL seems to be an overstatement (maybe some misplaced overenthusiasm being shown by O/RM proponents).
   3. O/RM technologies such as nHibernate and Linq provide caching and hence are good for execution.
      1. This statement is only partially correct. Caching helps only my Select statement performance. What about Inserts and updates?
      2. If I have a method which repeatedly updates or inserts data into some of my tables, then caching is definitely a performance degradation point because of the two updates it will need to do (once in cache and once in memory). And not to talk about the amount of degradation this will cause when I have my application running on a distributed environment (Ted Neward talks at length about this scenario).
   4. Maybe there are more advantages to using a O/RM solution (will update this post in case I find any serious advantage that I missed here). But, overall I could find only one advantage which makes me take a serious look at O/RM (i.e. the ability to ship to multiple vendors irrespective of their DB Systems).

Now coming to the disadvantages of using a O/RM solution:

I think I cannot write more than what Mr. Ted Neward has written in his article. It is an excellent article which talks about design considerations (which most of tend to ignore while designing our applications ) and where O/RM falls short in addressing the problems that it tries to address.

I wanted to end this post with the following statement,”O/RM is a good solution for addressing issues where we do not have to bother about inserts or updates. Hence we might use a solution where O/RM is used for addressing some of our issues and use stored procedures where performance is the key requirement.“

But after reading the last statement of Mr. Ted Neward’s article –”Unfortunately, as the history of the Vietnam War shows, even an awareness of the dangers of the Slippery Slope is often not enough to avoid getting bogged down in a quagmire. Worse, it is a quagmire that is simply too attractive to pass up, a Siren song that continues to draw development teams from all sizes of corporations (including those at Microsoft, IBM, Oracle, and Sun, to name a few) against the rocks, with spectacular results. Lash yourself to the mast if you wish to hear the song, but let the sailors row.“– I have decided to refrain myself from giving any suggestions. But whatever it might be, I do like one aspect of O/RM and hence I will definitely take a deeper look at it ( especially Linq ).

Pratap.

The first day I joined IIIT-Hyd I wandered around and finally reached the library. Our seniors were still in the process of vacating the old lab (taking backups of their data etc) and we first years still did not have our machines allotted to us. Library was the only place where we had FOUR computers which we had access to.

I sat in front of one such Computer and was puzzled with its looks. It had a normal monitor but a very small CPU. I was a bit new to the Computing field and at that time the things I knew about computers was floppy disks, CPU, monitor, keyboard and mouse. And a CPU was supposed to look big and it must have a floppy disk. But the one in front of me did not fit that definition. Then I switched on that box and it started flickering a GUI which was even weirder. It was something that I had never seen before and the only GUI I was ever familiar with was the Windows 95/98 GUI. And the one was confusing me to the core. I assumed that since it is IIIT and the CPU is much smaller than a normal one, it should be a very very sophisticated box which must be in market .

But two days later I came to know that it was a CLIENT and that the actual Server was in our Server Room. The server was running Linux and what I was seeing as the GUI was Gnome (if I remember correctly) I worked on that box for around two weeks till we got our lab machines and then we shifted to working on our lab boxes. But why I am I writing about this year here?

These days I am hearing a lot about cloud computing. In many ways that idea to me seems to be an extension of the old Server-Client architecture on a grander scale. With network bandwidth increasing manifold over the last few decades I think technology has turned a full circle now. The dumb terminals of the 1970s and 80s are coming back with a vengeance ( though may not be as dumb as the original dumb terminals ) in the form of NetBooks and Mobile PCs. The MainFrames are getting replaced by entire DataCenters. Just as the original mainframes sold computing power to people who needed some (to develop their apps) for whatever reasons, datacenters are going to sell Computing space for application vendors.

    The idea is captivating enough. Imagining the scale by which we can cut operation costs for companies which invest in their own hardware space for their websites or any other server usage is just mind boggling. Most SMEs (small medium enterprises) will benefit with the cloud. And not just that, future developers just need an idea and a little amount of money… you can write the software and invest your pocket money on server space and get your app onto the cloud. This idea in itself is very enticing… the developer/investor need not ever think about how to invest in hardware, broadband connectivity, internet space, 24X7 availability (the biggest problem) and how to achieve scalability in hardware for higher loads. By cutting down so many operating variables from the mind of a investor/developer, cloud is going to just revolutionize the way we see the world.

Pratap.

Cursors. If you are working on databases then there is no way you could have missed this term. The only feature which allows you to work per row (by looping) instead of doing a batch operation. If you are a novice to database development then this is how you will try to solve every database problem (Because this is the way we are accustomed to writing programs using our regular programming languages such as C/C++)

How should you work with databases then?
Set based operations. That is how you should solve your problems. What is this term ’set based operations’. You may have come across this term at many places and may not have understood it at the first glance. The basic idea is very simple. Consider the following case:

1. You have a big customer table with three columns. Id, Name and Telephone number
   CREATE TABLE Customers(
Id INT,
Name NVARCHAR(256),
PhoneNumber NVARCHAR(256)
)
INSERT INTO Customers(Id, Name, Phone)
VALUES(1,'Pratap1','1000')
INSERT INTO Customers(Id, Name, Phone)
VALUES(2,'Pratap2','1001')

2. Let us think about a hypothetical case where the Telephone numbers for a particular set of thousands customers have changed.
3. Your task is to identify those customers and update their telephone numbers.
4. If you are a novice, your first solution would look something such as using a .NET datareader. Which is where cursors come into the picture. E.g.:
5. Assume that the Customer Ids to be updated with Phone numbers is stored in a temporary table called @TempCustIds
   1. Now, if you use a cursor your code segment will look something similar to this
      DECLARE @TempCustIds TABLE
(
Id INT,
Number NVARCHAR(256)
)INSERT INTO @TempCustIds(Id, Number) VALUES(1, ‘1002′)DECLARE FetchCursor CURSOR
      FOR SELECT cust.Id, cust.Name, tci.Number as NewPhoneNumber
      FROM Customers as cust
      INNER JOIN @TempCustIds as tci on cust.Id = tci.Id

      OPEN FetchCursor

      DECLARE @CustomerId INT, @CustomerName NVARCHAR(256), @PhoneNumber NVARCHAR(256)

      FETCH NEXT FROM FetchCursor
      INTO @CustomerId, @CustomerName, @PhoneNumber

      WHILE(@@FETCH_STATUS = 0)
      BEGIN
      UPDATE Customers SET PhoneNumber = @PhoneNumber WHERE
      Id = @CustomerId

      FETCH NEXT FROM FetchCursor
      INTO @CustomerId, @CustomerName, @PhoneNumber
      END

      CLOSE FetchCursor
      DEALLOCATE FetchCursor

6. But the first question you should ask yourself is, ‘Does this require a cursor? How can I avoid one and how should I actually write this query?’
7. The answer to this problem is very simple. Write a bulk update statement which updates all the required items in one shot. How? Consider the following query:
   1. UPDATE cust SET cust.PhoneNumber = tci.Number
      FROM Customers as cust INNER JOIN @TempCustIds as tci on cust.Id = tci.Id
8. Now how small is the this code compared to the first CURSOR code and check how fast it is compared to the CURSOR used code.
9. So, the first lesson is always think in terms of bulk operations (i.e. how do you achieve updating/selecting/inserting a batch of items at once). Never think in terms of one operation at a time. That is always going to slow things down (drastically!). This is what people mean when they say you have to deal database operations in SETS.

What about CURSORS?

There are people who don’t understand the real pain of using CURSORS. When you see such code with CURSORS thrown around, the first thought that should cross your mind is performance. CURSORS almost got me to a point where the entire application was about to sink to unseen depths because of the amount of performance issues it caused. It has been three years since I have ever tried using a CURSOR and would never use it in future too. I have somehow grown allergic to them.

What are the alternatives then?

As we all know, there will be times when we face a situation where we have to do a element by element processing.  So, are there any alternatives to using CURSORS? Fortunately there are . It is just that you need to evaluate the difference in performance yourselves.

Click here for an article which shows the alternatives to using CURSORS.

I will try to update this post with some performance number comparisons between using a cursor and not using one. But until then I think this is a big enough post .

Pratap.

These days I want to blog, but do not want to blog about personal issues. Hence, I have chosen a neutral route by blogging on technical issues. I am continuing that trend by writing more tech stuff .

Stored procedures are something with which I have been living for quite a while now. Last summer when my friend Akhil visited me at office, I proudly showed him the database design I did and told him “Dude! You know I have written more than 60 stored procedures in a matter of 6 weeks . Most of the functionality and Business Logic of the application has been handled by me in these procedures”.

He immediately shot back by saying that “Bah!! that is a bad design. You have complicated your own job by including lots of business logic at the database layer. I never use stored procedures for anything. I make direct calls from my Java code. You might have slogged but you haven’t designed well!!”.

Wow, I was a bit taken aback by his statements. I have been designing databases and writing lots of stored procedures but never have I really given a complete thought about maintainability. I always knew where to fix when a bug occurred, so fixing any issues with my own code was never an issue. But, I did miss out this part on code maintainability. Also, in my group the tendency is to push as much functionality as possible into the databases, because people think it makes the application more performant and secure. So, I went back to the books and started reading about how good or bad stored procedures were. Over time this is what I understood.

The Good about stored procedures:

Stored procedures are good for security. You can properly isolate your data access/modification logic by using proper security rules/permissions at the database layer. This reduces the risk of data corruption/theft.

Stored Procedure is nothing but a collection of SQL statements which are pre-compiled and stored in a database. So, something written at the database layer does give good performance.

They are useful to enforce any checks (which you might not be enforcing at the table level using check constraints) which you might want to do before data is accessed/entered into the tables.

The Bad about stored procedures:
What is the bad then?

The argument that ’since a stored procedure resides at the database layer, application becomes faster if we write everything at the database layer’ is absolute rubbish. There are ways and means by which you can get your middle tier or web layer to work much faster than always making a database access for every minor detail. Will give more details on how you can cache things at the MT/Web layers to get greater performance gains. But again, that does not mean you should write code like how my friend suggested. Which is writing SQL Statements in Java/C# and invoking them from a function. That is another extrememe which is very bad for performance and security.

So, What is the correct approach?

The first and foremost principle according to me is design. You need to design your tiers properly and decide what to do where. Business logic is best restricted to the Middle tier. Writing Business logic in stored procedures just complicates everything. Putting it into stored procedures is not going to give an iota of performance improvement because we unnecessarily end up with:

1. Complicated code
2. Less modularity in coding (as SQL does not provide as much flexibility for coding as C# or Java does)
3. Larger stored produres => Many SQL Statements. Which directly leads to locking and blocking issues by increasing our transaction times. On top of it sometimes SQL Server will unnecessarily spend time recompling stored procedures for unnecessary segments of code which may or may not be used during every call of the procedure. Also reading a larger proc always increases reads.

From the above statement there is one rule which I say everyone should identify. Differentiate between data access and business logic. Do not combine both. Restrict your Business logic to tiers above the database and concentrate data access/modification logic to the db layer. You will get much better performing procedures and also you can ensure proper security for your data. Also the overall performance of the system will go up because you aren’t keeping SQL server busy doing what it is not actually designed to do.

So, another guideline is proper design. Have a proper understanding of what SQL Server or other environments provide, to make us write better manageable and high performing systems. For example, some developers do not know that SQL Server provides notification based cache updates. Which means it supports some stuff which is highly useful. I have noticed people refuse to use such useful features by talking about fuzzy scenarios. So, first get your use-case scenarios correct and then you will have a smooth design flowing out. Do not overdo the use of any useful feature (just because SQL Server provides it) and end up complicating your life. Use them as per your requirements and design well.

I think I have already given lot of gyan in this post. Ending it here .

Pratap.

Because of privacy concerns I had stopped blogging for a while. But after this issue really bugged me for a few days and finally when I managed to get a work around for this issue, I thought that it would be a good idea to blog about it, so that others who face the same issue might not have to face so much of a headache as I did.

So what is that issue?
We were migrating our existing DTSX packages to SQL Server 2008 and there was one package which read from the OLAP Cube and processed data and inserted the processed data into our OLTP database.
So, the previous developer had used ADO .NET connection to connect to the Cube and it worked well with SQL Server 2005. When the issue of migrating this to SQL Server 2008 came up, the ADO .NET Connection started failing.

The exact error was something similar to “Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The “component “ADO NET Source” (43)” failed because error code 0×80004002 occurred, and the error row disposition… Error: The component “ADO NET Source” (43) was unable to process the data. Pipeline component has returned HRESULT error code 0xC0209029 from a method call… SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component “ADO NET Source” (43) returned error code 0xC02090F5. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.”

Upon serious searching on the web I came across this blog post which talks at length about how ADO.NET connectors do not work with SQL Server 2008. As it was suggested in the article I changed all the ADO.NET connectors to OLEDB sources and after that I again got another issue.

As soon as we start debugging the package or running the package, the SQLDumper.exe gets invoked as soon as the execution reaches the OLEDB Source. Looks like an access to the Cube wasn’t happening, the DTExec started crashing. I sent multiple mails to the concerned teams and also put up a post on forums.microsoft.com. But none of them gave me good work arounds or any ideas on how to solve this issue. To check whether it helps I also installed SQL Server 2008 Cumulative Update 3 and even that did not work.

Then finally I chanced upon a workaround that was suggested by the SSIS team in 2006 for a similar problem with SSIS 2005.

So, I solved my problem by doing the following:
1. Creating a linked server on my database machine (and making that link point to the analysis server).
2. Using a OLDEDB connector (in my DTSX package) pointing to my database server (instead of the analysis server)
3. Using a OPENQUERY as a wrapper around the MDX query which was previously being used in the OLEDB connector to the analysis server.

This issue ate way two to three days of my development time and created a big headache. I hope this post helps all those people who face the same issue to overcome it.

Pratap.

Been trying to learn photography for sometime … will try to put up some pics on this blog… I am getting a feeling that my blog is getting a tad boring these days and the enthu to write is also going down… because the number of hits to my wordpress blog is considerably less compared to the number of hits on my yahoo blog. Anyway let me see if the hit count increases overtime .

Pratap.