How to address Schema Migration for Dummies

  1. Be very clear in what your PersistentEntity (an entity that directly corresponds to a db table) is and separate it from your Business related entities that may or may not have one to one relationship with the tables. Ideally, Business Entities should be defined on a need basis, however PersistentEntities should be defined as soon as your schema / data model design is ready for the first iteration of development.
  2. Once you create entities, you will annotate these entities, and I haved shared an example of such entities. These annotations should follow JPA annotations standard and not Hibernate annotations, because that way you will avoid vendor lockin, and be following a standard. JPA is a standard developed by Java Community and has JSR (Java Specification Request) which is implemented by Hibernate, J2EE, TopLink or any other middle-tier OR implementation.
  3. Add Hibernate Synchronizer plugin to Eclipse (you can also use the schem exporter command line utility, which you will find docs on RedHat Hibernate website). I highly recommend using Eclipse plugin because it makes the process visual and easy to follow.
  4. Use Hibernate Sychronizer to Export the Schema based on the PersistentEntities, this will require you to configure your persistence.xml file, which is usually placed in the class path, inside a jar or directly under the class path  i can share some examples for demonstration.
  5. The schema exporter works by pointing to your data base instance and generating all the tables that you have defined as PersistentEntities, taking care of all the constraint definitions, indexes etc. Will share a sample of that too.
  6. What you now have is the v1 of your db. This is ready to be used in your application by defining DAO or Data Access Layer, which abstracts out the operations performed on PersistentEntities. Note: As long as you have a good abstraction around DAO, your persistent entities will mostly remain unchanged until you decide to add more columns or tables to your schema.
  7. Now for v2, if you are adding more tables and columns, you will start with modifying the PersistentEntities directly and adding news ones where required. Reminder: All the entity relationships are also defined using Annotations, as you already are aware, and hence adding new tables only requires some new relationship annotations to be added to exisinting entities if they are going to have a new relationship with the newly added entity.
  8. After you are happy with your v2 db schema design and PersistentEntities created thereof, you will point to your Development DB server and generated the new schema based db instance.
  9. In this step you will need to have a tool that can compare db instance v1 with db instance v2 and generate a migration script for the DDL part. For data that needs to be migrated, you will have to write your own migration scripts. Note: If you are using MySQL then you are lucky because MySQL Workbench has great tools for schema comparison and migration DDL auto generation.

PersistentEntity

package com.rishik.hibernate.entity;

import java.io.Serializable;

import javax.persistence.CascadeType;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.JoinColumn;
import javax.persistence.Lob;
import javax.persistence.ManyToOne;
import javax.persistence.Table;

import org.hibernate.annotations.Generated;
import org.hibernate.annotations.GenerationTime;

/**
* This is an object that contains data related to the file table.
*
*
* table="file"
*/

@Entity
@Table(name="file")
@SuppressWarnings("serial")
public class FileEntity implements Serializable {

public static String REF = "FileEntity";
public static String PROP_TYPE = "type";
public static String PROP_ID = "id";
public static String PROP_USER = "user";
public static String PROP_CONTENT = "content";
public static String PROP_WORKSPACE = "workspace";
public static String PROP_CREATED_DATE = "createdDate";
public static String PROP_LAST_EDITED_DATE = "lastModifiedDate";
public static String PROP_LAST_EDITED_BY = "lastModifiedBy";

// constructors
public FileEntity () {
initialize();
}

/**
* Constructor for primary key
*/
public FileEntity (java.lang.Long id) {
this.setId(id);
initialize();
}

/**
* Constructor for required fields
*/
public FileEntity (
java.lang.Long id,
java.lang.Long userId,
java.lang.Long workspaceId) {

this.setId(id);
this.setUser(user);
this.setWorkspace(workspace);
initialize();
}

protected void initialize () {}

private int hashCode = Integer.MIN_VALUE;

// primary key
private java.lang.Long id = 0L;

// fields
private UserEntity user;
private WorkspaceEntity workspace;
private java.lang.String content;
private java.lang.String type;
private java.util.Date createdDate;
private java.util.Date lastModifiedDate;
private UserEntity lastModifiedBy;

/**
* Return the unique identifier of this class
* @hibernate.id
* generator-class="native"
* column="file_id"
*/
@Id
@GeneratedValue(strategy=GenerationType.IDENTITY)
@Generated(GenerationTime.INSERT)
@Column(name="file_id",insertable=true,updatable=false)
public java.lang.Long getId () {
return id;
}

/**
* Set the unique identifier of this class
* @param id the new ID
*/
public void setId (java.lang.Long id) {
this.id = id;
this.hashCode = Integer.MIN_VALUE;
}

/**
* Return the value associated with the column: user_id
*/
@ManyToOne(cascade=CascadeType.ALL)
@JoinColumn(name="user_id")
public UserEntity getUser () {
return user;
}

/**
* Set the value related to the column: user_id
* @param user the user_id value
*/
public void setUser (UserEntity user) {
this.user = user;
}

/**
* @return
*/
@ManyToOne(cascade=CascadeType.ALL)
@JoinColumn(name="workspace_id")
public WorkspaceEntity getWorkspace() {
return workspace;
}

/**
* @param workspace
*/
public void setWorkspace(WorkspaceEntity workspace) {
this.workspace = workspace;
}

/**
* Return the value associated with the column: content
*/
@Column(name="content")
@Lob
public java.lang.String getContent () {
return content;
}

/**
* Set the value related to the column: content
* @param content the content value
*/
public void setContent (java.lang.String content) {
this.content = content;
}

/**
* Return the value associated with the column: type
*/
@Column(name="type")
public java.lang.String getType () {
return type;
}

/**
* Set the value related to the column: type
* @param type the type value
*/
public void setType (java.lang.String type) {
this.type = type;
}

/**
* @return the createdDate
*/
@Column(name="created_date")
public java.util.Date getCreatedDate() {
return createdDate;
}

/**
* @param createdDate the createdDate to set
*/
public void setCreatedDate(java.util.Date createdDate) {
this.createdDate = createdDate;
}

/**
* @return the lastModifiedDate
*/
@Column(name="last_modified_date")
public java.util.Date getLastModifiedDate() {
return lastModifiedDate;
}

/**
* @param lastModifiedDate the lastModifiedDate to set
*/
public void setLastModifiedDate(java.util.Date lastModifiedDate) {
this.lastModifiedDate = lastModifiedDate;
}

/**
* @return the lastModifiedBy
*/
@ManyToOne(cascade=CascadeType.ALL)
@JoinColumn(name="last_modified_by")
public UserEntity getLastModifiedBy() {
return lastModifiedBy;
}

/**
* @param lastModifiedBy the lastModifiedBy to set
*/
public void setLastModifiedBy(UserEntity lastModifiedBy) {
this.lastModifiedBy = lastModifiedBy;
}

public boolean equals (Object obj) {
if (null == obj) return false;
if (!(obj instanceof FileEntity)) return false;
else {
FileEntity pOFile = (FileEntity) obj;
if (null == this.getId() || null == pOFile.getId()) return false;
else return (this.getId().equals(pOFile.getId()));
}
}

public int hashCode () {
if (Integer.MIN_VALUE == this.hashCode) {
if (null == this.getId()) return super.hashCode();
else {
String hashStr = this.getClass().getName() + ":" + this.getId().hashCode();
this.hashCode = hashStr.hashCode();
}
}
return this.hashCode;
}

public String toString () {
return super.toString();
}

}

A/B Testing – Knowing what the User Wants

A/B Testing?

Have you ever wondered how the Social Games like Farmville and MafiaWars become so viral and engaging? How Google, Amazon, eBay and Zynga are able to present to you the content at the right time and with the right keywords to grab your attention?

There is a User Testing process known as A/B Testing that is employed by many giant IT companies that run their business online. It started as an email-marketing trick, to present variations (single-variable at a time) of the same content to different cohorts of users and collect the response and the change in response based on those variations.

For instance, if 10 users were sent emails about a flower shop in downtown San Jose, 5 of the users might see the picture of the flower shop at the top and 5 users will see the same picture at the bottom. Based on which group of users chose to read the mail and follow the links more would then be collected, using the parametrized urls embedded in the email, to determine which approach was working better than the other. And that information would then be used to create email templates that would have higher chances of being followed by the user. This stochastic analysis allows the data analysts to create models based on different variants of the component that are being tested and use these models to incorporate attractive features to the marketing tools and eventually the products that are being marketed. Here is  demo to demonstrate what is actually tested,  ABTestDemo.

The Science

To make it work, users shouldn’t really know that they are being tested on, as knowing that may affect their natural reaction. Thus, only one of the two options are presented to a user at a time and user’s reaction is captured to measure effectiveness of that option for that particular user; a collection of responses for a larger user base allows analysts to determine how that cohort generally feels about the feature. ABTests are very similar to psychometric tests except the goal is not to understand a particular user better but to establish behavioral trends in a large group of users.

Having said that, it is still a huge task to identify what should be and can be tested, how the variations should be parametrized, which variations should be used for which cohort and how to interpret the results collected for each variation used in the tests. Since both the scale and the magnitude of ABTesting can easily become intractable, ABTesting and the conclusions drawn from the results are basically part of a stochastic process. What that means is the conclusions are not definite but probabilistic. Or in other words, what a test like the one shown above in ABTestDemo can tell us deterministically is 8 out of 10 users like Option B more than Option A, however it may not be able to tell us why 80% users like one option over the other since the users that are making the choices do not consciously make a choice rather they are subconsciously being driven towards or away from the conversion goal (where conversion may mean different things to different applications or websites.) Therefore, the improvement from ABTesting is dependent on the testing team coming up with really compelling options for the users. If ‘Option A’ and ‘Option B’  of a test end up evoking the same or similar emotions, the test can not really yield any worthwhile results. So, choosing the right options for the tests and choosing the right tests becomes critical to the success of ABTesting.

The Technology

I came across ABTesting while working as the SDK architect at Playfirst Inc. I was developing the SDK and was assigned the additional responsibility of including ABTesting support in the games, by means of SDK. What I had was a hashing algorithm for creating the basic percentage based buckets to allocate the test variants to. The architecture was very simple, there was a service written in PHP that would be invoked at the time of deployment to load the A/B variations and rules of distribution for the percentage of users getting allocated to one or the other bucket for each test. These rules once created per persisted in the Memcache, so that when each user logged in to the game, their hash would be calculated using their unique id and some other input parameters. The calculated hash would then be used to place user under one of the two A/B buckets based on which percentage distribution range they fell in. This allocation map for all tests the user would undergo was then stored against user’s id in the Memcache, so on subsequent login, the valid tests would not be recalculated. However, the cached maps were invalidated every time the tests, the distribution or the rules changed on the server side. Once calculated the bucket in which user was placed would be used as an additional parameter for all the subsequent requests from client to server, so that the server will send metadata that was applicable for the user’s allocated bucket option. The client would also have the intelligence to present features (being tested) differently to the users and measure their responses.

Conclusion

I am a big fan of this method of testing and collecting data because this approach favors an unbiased and natural process of selection of a stronger candidate. I think this is the most natural way of evolution and is therefore more organic, since no abrupt undesirable or hard-to-deal-with changes are introduced to the users. Quickly changing products cause more nervousness than confidence in users. Even thought it can be a slightly slow and cumbersome process, it promises a much deeper insight into user behavior and how the product is perceived generally. This method is also very tricky to implement because identifying the areas to be tested and tests to be defined takes enormously wide perspective and can become an issue of disconcert between the team members designing and implementing it. Also, I haven’t come across any frameworks or cookie-cutter templates for implementing ABTesting I guess the reason is that each product has different needs and is implemented differently, so it will take a while before patterns are identified and frameworks are created.

References

  1. Wikipedia – http://en.wikipedia.org/wiki/A/B_testing
  2. http://www.smashingmagazine.com/2010/06/24/the-ultimate-guide-to-a-b-testing/
  3. http://thinkvitamin.com/design/human-photos-double-your-conversion-rate/
  4. http://www.topnews.in/healthcare/content/21929garlic-good-high-blood-pressure

JavaScript 2.0 – A New Avatar

Java Script for objects: Brought to you by ECMA.

Now this is something that was pending for a long time now. Twelve years since the birth of JavaScript and no major or minor updates to the language. Imagine how much of value it must have offered to the entire web developer community to have survived in the software world without changing for twelve years.

As the creator of JavaScript and currently the CTO of Mozilla Foundation , Brendan Eich said,”It is not a revolution, just an evolution of existing capabilities.” It sure has been long due now. Especially, with the AJAX and Web2.0 taking WWW to a different paradigm altogether. Making it essential that the foundation of the new web be stronger and reinforced with as much steel as possible.

I would be looking forward to see how it evolves. The primary challenges would be to keep it as simple as it has been and still provide an OO environment for building RIA.

Packaging and scoping are some of the mandatory requirements for the JavaScript 2.0 to keep it rolling. And another challenge is the ascertain that the JavaScript loaded on the browser is not a security risk for the computer, because the internet browsers, as we know them today, don’t provide proper sandbox environment like a JVMs do.

There is a lot that would be expected out JS2.0 and I am sure, like before, it is going to deliver.

You can track the progress of JavaScript 2.0 or as the project Code name says ECMAScript from this link. ECMAScript

Torrent over RIA…Cascading Revolution!

Can Do: A New Paradigm for content sharing

A year and a half back a friend of mine, who is doing his research on biotechnology, explained to me a new way of sharing files that was taking heat and that could actually make things far more quicker in spite of the bandwidth variations across the internet.

Bit Torrent is a very innovative protocol designed to make file sharing over internet much easier and less bandwidth-intensive. Amazing technological advancement in the area of content distribution with shared bandwidth.

The concept of fragmenting the target files across a host of computers connected to the internet makes the entire idea of “high load on single server” obsolete. The connected computers distribute the fragments by downloading them from other computers and then making them available for upload for those computers that don’t already have the fragment.

This enables sharing between a large number of peers possible without increasing the load drastically on any single machine. And thus making it far more faster than a normal download over an FTP or HTTP would take for the same file size.

The downloads and uploads however still remain bandwidth dependent as everything else is on internet but in this case the dependency never leads to a bottle neck. The flow is constant and the load shared therefore reducing the chances of having single point of failure. Bit Torrent also enables non-continuous downloads inherently because the continuity can never be assured over internet given the fluctuating nature of the bandwidth.

Desire: Ease of Usage and Friendliness

To avail the power of Bit Torrent based file sharing, you would normally require a client side application that helps you in performing download management tasks viz. Bandwith Allocation per download, prioritizing downloads, pausing and resuming, seeding etc.

And to those of you who use internet extensively, installing yet another application sounds to be an encumbrance. Organizing all the applications on your desktop, then updating versions, re-installation, firewall limitations etc. are some of the common problems one faces while using tools like Torrent clients.

How one wishes there was a way of putting all these applications on top of another layer of organization within your desktop that would manage all these issues, instead of your having to sit once every week and arrange stuff on your desktop manually.

Alternative: Desktop Rich Internet Application Platform

Having a RIA platform opens a great deal of opportunities for both developers and end-users. Developers find it easy to experiment with their ideas and offer their services over a customizable application development platform. End-users on the other hand benefit from a constantly growing community effort in bringing out function rich applications.

The RIA platforms and the desktop web has caught interest of major software vendors in the industry, to name a few Google and Adobe and they are in a race to offer these platforms out of their angelic desires to make the end user ever more equipped and enabled to leverage the strength of internet and web technologies.

Pramati, a relatively small fish in the huge ocean, on the other hand has already arrived with a solution that is simple, elegant and as powerful as your imagination. Dekoh in it’s entirety has altogether different view about how the world to come will use internet.

As I have been talking about Dekoh in my previous posts, so I won’t bother you with what Dekoh is. However, Dekoh can be used in places where no one could have ever imagined. Dekoh has evolved considerably in the past few months and its capabilities are growing by the day.

For a torrent service and client application all you need is a basic web server with trackers and clients that can communicate over the internet. Dekoh provides you with a personal webserver and there is a Central Application Server, both of these combined can add broader tracking facilities for Bit-Torrent files.

Dekoh architecture also enables you to work from behind the firewalls which means that it is possible that your IM services may be interrupted or your access to the internet may be limited but your downloads will never stop as long as you have Dekoh with you.

Wondering on these possibilities, the day is not far when you might actually hear about Torrent being available over Dekoh platform, making end-users’ lives much more simpler and organized.

Torrent over a Desktop based RIA platform, in my opinion will make a killer combination and I will be following up on it more. You keep tuned in to know more.

Google is Google after all…

Blogging has grown from a personal interest to a full time profession that can get you paid heavily. Weblogs are being used for Social Networking, Business Networking, Marketing, Advertising, Solicitation and what not. But what do you expect from a weblog of Google, not something that I found out recently.

All bloggers are burning their finger tips to make their presence felt over the internet. And then it happens all of a sudden that you come across blogs like these, http://googlegearsblog.com/2007/06/07/dekoh-this/ .

This makes you realize that there is someone ahead of you capitalizing on your efforts and getting further ahead. I don’t feel there is anything unethical in this practice but it is a bit too much of a show off for Google. They don’t even want to write their own blogs about their own products. They have created a mashup that pulls data from the weblogs that contains articles on Google or its products. Now that is cool and that is why “Google is Google after all.”

The link above shows a post made out of my post in which I had mentioned Google Gears. This site is an auto-blog that crawls the web for blogs and other informational sites, filters out content that contains people’s opinion about Google and its products, and posts it here, with no effort required whatsoever.

Using the Technorati’s ping facility, this blog keeps updating itself regularly and pinging it to Technorati, making it hot on the tag cloud. Google has really gained a lot of headway through its search and related products and now this is another use, they have found, of their searching capabilities.

I am not sure whether they plan to do it as a practice for all their products or it is just an experiment for Google Gears, but it sure feels very unGoogle like to me.

At the same time, I can’t deny the fact that this practice makes sure that blogs like mine get a fair chance for gaining visibility. This might even get me more hits and increase the chances of my opinion getting heard by a wider audience. So, in short even though I am not impressed by this blog by Google, I am still curious to see what it will mean for Google and the rest of us, eventually.

Gizmoz: Technology Innovation OR Wastage

Gizmoz gives you the facility to build a expressive animated face out of any frontal snap you would like to animate. A cool tool for kids, but I failed to see the importance of this tool in its current stage of evolution. It’s likely that We might get to see a better use case than the ones this website is advertising.

I feel the engineers behind this tool should take their skills, imagination and time more seriously.
Two points on Gizmoz –

It doesn’t really give you a well-finished animation.

You can’t use these animations on your answering machines and be takenly seriously by the caller. Besides, I didn’t understand why do you need an animation for your answering machine.

Buzz or Biz???

If you read my first post completely you might have noticed that I signed-off with an odd question.

Let’s see which comes first, the chicken or the egg?

You may call it a foresight, divine intervention or coming events cast shadow thing!

I am in fact struggling with a chicken or egg situation here. Y te presento, mi predicamento:

What comes first in today’s fast changing and hyperactive software industry.

The Buzz or The Biz?

For the uninitiated or intellectually challenged, What makes a product or service big in today’s effervescent market, the business proposition that a company offers or the buzz they manage to generate?

It took me nearly 6-7 years to realize what Google was offering through their search engine was nearly unparalleled by anybody before or after. I have been reading many websites talking about semantic webs and specialized searches for different type of resources that are search candidates. But, there is no other Google when it comes to finding something on the internet. And yet I started using the term Google as a verb “Did you Google for xyz?” instead of a noun “Did you search for xyz on Google?” just about a year back. So Google-ing has become synonymous to searching stuff on internet. And on a personal level, I feel any product that becomes synonymous to an activity in our day to day lives, has created a “Buzz” or has become a “Buzz”.

So, in Google’s case, Buzz came after Biz. Yet sometimes Buzz comes before Biz. Like an Apple iPhone. As far as end users like me are concerned, all we know is that it is going to be something out of this world. And we like to believe that because Apple has established itself as a Style Icon in the world of technology for their elegant designs and catchy outfit. And the Buzz might in fact take iPhone’s sales to the sky even before people realized the limitations and drawbacks (if any). So, Apple’s iPhone might actually sell-off more pieces out of buzz than the real biz offerings.

This leads me to the profound question of Chicken or Egg? Or in my case Biz or Buzz???

I would bet my money on Biz. And yet I can’t ignore the strength of Buzz. I am working on creating a Buzz about a Biz. And let’s see which one yields better results for me…

Dekoh This!

Pramati launched Dekoh at Web2.0 Expo and we have been growing in our user base ever since. Here are some points you should know…

  • It is not a browser plug-in.
  • Yes! you have to install it on your computer but it takes less than 5 minutes for the framework installation on a high-speed network connection.
  • It is not the same thing as Google Gears or Adobe’s Apollo, it is very different and is more focused on bringing lay users the facility to host applications and content on their home computer, only for small scale web application hosting, social networking and collaboration.
  • Works out of a web browser, so no pain of learning how to operate on some new proprietary user interface.

Evaluate Dekoh, if you want all these things:

  1. Personal Hosting: A platform that enables you to build custom web applications to be used within your community and host them from your desktop.
  2. No-Upload Sharing: Want to share content with your friends, family and/or peer group without uploading any stuff anywhere.
  3. Personal Audio/Video Streaming Stations: You want to play music or videos stored at your home desktop anywhere in the world.
  4. Work Off line: You want to maintain to build an application that connects to Web Service, synchronize your local data store and then allows you to work on that data even when you are off line. (like in a basement, in flight, or while you are out camping.)

Benefits of working on Dekoh Platform:

  1. Open Source: You don’t have to pay anything but attention (to your idea)!
  2. Free Support: Our engineers will help you customize the platform (if you really need it), application support (for configuration and maintenance related issues).
  3. Creative Freedom: You can build almost any sort of application that needs a web server to run it. You imagination is your limit.
  4. Web 2.0 Toolkit: You can use our widgets and request for other components that are related to Web 2.0 to add flavor and flexibility to your and your community’s experience.
  5. Dev Kit: The best part about Dekoh is that a Dev Kit for building basic applications and deploying them takes under 30 seconds.
  6. Powerful Application Server: You will be leveraging the power of our enterprise application server, which has been condensed and made available to you for free with almost the same robustness, scalability and extensibility.
  7. API for extension: If you have some experience with extending API’s, you will know what we are talking about here. The potential to enhance the whole platform to your personal advantage with no cost involved.
  8. Be Trendy: It’s a cool app, which will make your desktop look cooler than before. The industry is moving towards offline web applications, and giants like Google and Adobe are the best benchmarks for understanding the trend.

Hello World – I am here!

There can’t be a more apt way of entering the Java world than “Hello World!”. Those magical words can transform about the most ordinary lives. At least for me they did.

I am relatively an infant in the Java world, but I am a precocious infant so I will say whatever I think is right and you have only two options, you agree with me or you die.

Java technology stack has grown from strength to strength in last decade and today it has reached a level of maturity that inspires confidence that the world could indeed depend on software. That intangible, quirky stuff some nerds keep punching in on their keyboards and that can do some wonderful stuff only we don’t know how it does that.

My ride through the software world allowed me to witness the prehistoric programmig languages like Fortran-77 and Pascal but as I moved ahead it has only gotten better with time.

In the life of a programmer, the real transformation occurs when one moves from straightforward procedural languages to the world of Object Oriented Programming, which kinda adds a realistic paradigm to the world of software. The complexity of objects in the real world and their relationships are translated verbatim in the OOPS world and that is when things start getting better. It’s like a boy turning into a man. Both the programmer and his code enter a new level of maturity and capability.

So, who am I? My name Rishik, and I would like to think it is a unique name, however it is no more. I have worked as a programmer for last 6 years now and worked on different sets of technologies. Currently, I am having an affair with Java Technology Stack. And I can’t express how rewarding this relationship has been right from its inception. I work with Pramati Technologies, and I have recently added another role to my portfolio, which is “Relationship Manager”. Apart from being a developer I am now into business development for half of my time. And I started this blog to bring out the lighter side of Java.

Let’s see what comes first, the chicken or the egg!