Monday, July 5, 2010

ExtDirect GridPanel w/ Connect on NodeJS!

8/21 Update: I've updated the sample code to work with:
  • NodeJS v0.2.0+
  • Connect v0.2.4+
  • Mongoose  v0.0.2+
  • MongoDB v1.6.0
The Inspiration

So okay, earlier this year I stumbled upon NodeJS and MongoDB and I thought eureka! Finally, I can do front to back JavaScript from the display all the way to the DB and rid myself of all of the complexities and transformations that I had to deal with in my last major project.

That one was an ExtJS interface plumbed via DWR to a Java back-end, utilizing Hibernate to access a MS SQL Server DB. Hibernate has some awesome capabilities and DWR makes bridging the browser to server gulf a breeze (especially using, but I was so yearning for something more direct and simple. It seemed like such a hassle overall to get data back and forth.

The Rationale

When I found NodeJS I thought, sweet! I've got JavaScript and JSON on both sides, so no more extra overhead needed there. I had already come to understand and appreciate asynchronous behavior so Node's message hit home there. And when I found MongoDB I was ecstatic! No more relational to object mapping anymore! My objects (documents) are already in the database.

The Process

So, once I got my new laptop in May, I started down the path of educating myself on this new world whenever I found some free time. As with anything new, there are difficulties and bumps on the road. Since all of these server side components can only effectively be run on Linux (or OSX), I had to learn and setup an Ubuntu guest on VitualBox and install and configure all of the relevant bits before I could even start coding! That alone was many hours of fun there I can assure you... :) Fortunately for me, I had been using Ubuntu for at least a year on my previous old and sad laptop. So at least I had some experience navigating it and besides, Ubuntu makes Linux about as easy as it can be.

So with that all setup, I slowly took the next baby steps. I ran the obligatory Node "Hello World" app. Then by this point the ExtJS (Sencha) folks had just released Connect. I loved working with ExtJS, so I immediately jumped on that and ran all of the examples and thought how I could build out a working ExtJS example app? Since they didn't have an Ext.Direct provider, I first went down the JSONRPC path and reworked one of the ExtJS grid examples to pull data from my Node server using all of those components. But then I realized how much more work I'd have to do to get all of the other CRUD interfaces plumbed so I switched gears and went with Ext.Direct and built out my own Connect provider for Direct.

Once the basics of the Direct provider were working it was time to turn to MongoDB and complete the full data life cycle. While searching for the appropriate driver I found Mongoose which sort of claims to be an ORM, but since MongoDB already stores data as document objects, I would say Mongoose is more of a convenience layer that makes interacting with Mongo much less tedious and helps to make your code more clear and concise.

The Results

So I ended up with a slightly contrived example, but it does exercise all of the general CRUD type grid operations including sorting and paging.

The main application code itself is comprised of 3 files:
  1. CompanyGrid-ui.js - the Ext grid UI layout code (the view)
  2. CompanyGrid.js - the "business logic" extension of the view (the controller)
  3. app.js - the server side RPC data access layer (the model) and server.
Of course there are other files like the Testr.html which starts up the app and pulls in all of the required resource and library files (CSS, images and ExtJS).

In a more proper solution, the app.js would probably be broken into multiple files.  A quick rundown of this code reveals three main sections.  The first being the model definition for the Company class and the database connection and the MongoDB data initialization via Mongoose.

The second section includes the direct provider handler that manages all of the client data requests.

And the last piece basically starts the Node server via Connect and adds the appropriate middleware and configuration.

For this app, the server environment includes the following open source projects:
As of this writing, the Connect library does not include an Ext.Direct provider and I have hacked my own based on their JSONRPC provider.  In my configuration I have placed this directProvider.js file within the \lib folder under my application path on my server.

A consolidated zip archive of the project files including the direct provider and a layout of the project folders can be found at this forum post.  In my setup I have the connect folder symlinked under the project's lib folder.

Saturday, July 3, 2010

Ext.grid.GridPanel and column sorting...

Tip: Do not use the field mapping option when enabling column sorting on a grid and just rely on the ColumnModel dataIndex to properly map property values from your source data. The reason being is the field name is passed on the sort parameter field value to the server and not the mapped property name. So if you do map a field, then you will have to manage the field name on the server side injecting display logic into your operations layer.

Saturday, June 26, 2010

VirtualBox networking between host & guest

When I first setup VirtualBox 3.2.4 to host Ubuntu 9.04 on Windows 7, I had a NAT network adapter selected and soon found that my host could not see the virtual guest and the NodeJS server I was running. So after Googling I found a great post that directed me to to use the Bridged Adapter that would request an IP address from my router and play as an equal member of my network and all was good... but then later I happened to be offline and since there was no central DHCP server, my host could not see my guest once again! A little more Googling and experimentation later I found that the best solution for me was to setup a second Host-Only Adapter set to the VirtualBox Host-Only Ethernet Adapter. When the guest started I selected to enable the second adapter and so now my setup would work whether I was online or off! Yeah!!! :D

Friday, June 4, 2010

SAML and WSRP trials and tribulations

Well, this wasn't the first blog I thought I'd be writing, but since I just resolved a blocking production promotion configuration issue last night and the initial exception I ran into only turned up one forum post when Googling on it, I thought this might be a good one to post...

Background: We run several clustered BEA 10 domains at work.  Each run 2 nodes each on 3 boxes.  At the primary data center we have 3 domains running on those 3 boxes and at a secondary data center in another state we have only one active domain with the same configuration.

In this this case I developed and deployed a WSRP producer EAR at the secondary data center and I needed to register that producer on one of the domains located at the primary domain.  The steps to setup SAML and WSRP are not difficult and the documentation is pretty easy to follow. However, when I first configured this setup in the test environment I ran into the following exception:
com.bea.wsrp.faults.TransportException: Security token failed to validate. weblogic.xml.crypto.wss.SecurityTokenValidateResult@5d70680[status: false][msg The SAML token is not valid.]
After tweaking and adjusting everything I could think of, I could finally get it to work if I "touched" the configuration, but then it would eventually stop working or if I bounced the nodes the exception returned immediately. I opened a support case with Oracle, but they were not able to solve it before I happened on a solution myself. What I discovered is that in our configuration I had to check (set to true) the "Include Keyinfo" option on the Relying Party defined on the SAMLCredentialMapper. Why so is still a mystery to me...

Now when it came to the production promotion, I thought, based on my test deployment and configuration, I had a firm understanding of what I had to do if not exactly why. And it turned out in the end I did, but I still ran into a speed bump! When trying to apply the "Issuer URI" and "Signing Key Alias" changes to the SAMLCredentialMapper I got the following error:
SAMLBeanUpdateListener SAMLCredentialMapperV2: prepareUpdate() failed with exception: provider SAMLCredentialMapperV2: SAML Key Manage failed to validate key (Assertions signing) configuration in the FederationServicesMBean
The one forum post I found did not give an answer except to manually change the setting in the config.xml and restart. So that's what I did! :) Everything came up as expected so I moved forward and then my old nemesis returned... I was seeing the token error again when trying to register the producer.

After engaging Oracle support again to no avail, I once again solved this one with a little help from my buddy in the Data Security group. Turns out the keystore files in the production environment were not configured with shared aliases (specifically the one I was using on the credential mapper) and I noticed that early on and dutifully used keytool to export and import the aliases to the other boxes in the cluster. I thought I was smart and had averted that problem before it started. But I wasn't smart enough...

In the end what I found is that the aliases were chained (visible when using the command: keytool -list -v) and had a private key. So my simple export/import did not bring over the second chained entry nor the private key. Fortunately for me, Google did find me a good solution in KeyTool IUI. With KeyTool IUI I was able to export the chain and the private key from one keystore and import them both into the other keystore files.  Once the aliases were properly set on each box, the producer registered perfectly and the content was there!

Here are the two screens I used to export and import my aliases:

Tuesday, June 1, 2010

Why is this here?!

The obligatory first post!

I've been programming for many years and some of the most helpful information that has either saved me time or saved my butt I've found on one blog post or another. So I thought I'd try and pay back the gift and blog on my own technical triumphs in case they may be useful to someone else...

Whatever subject I'm sure to blog on will be something I'm probably not an expert on since I was struggling to solve it myself and felt that it might be worth sharing and possibly if i'm lucky I might get some additional helpful feedback! :)