Data and the Web

Well, it's the last day of Web 2.0 Expo. If this turns out like the the first two days of the expo, it should be another great day of continuous conversations and demos.

Web 2.0 Expo is certainly a much larger show than the MySQL conference. There's simply an enormous number of people here, so the dynamic is quite a bit different. When we were at the MySQL show, it felt like hanging out with a close group of friends; at Web 2.0 Expo, it's more like being at a rock festival.

Although the scale is different, one constant is the amazing new technology that's on display. One of the eye-catching tools that we've enjoyed seeing at both shows is MindTouch's Deki Wiki. This is basically a next generation Wiki platform that really expands the idea of what a Wiki can be. For instance, it has both an API and the ability to host structured data. This is exciting for us, since it opens up the possibility of creating a “Data Wiki”– a platform for publishing and editing data in a collaborative way. Maybe the idea of a “Wikipedia for data” is not too far away after all.

Overall, it's certainly fun to be in the middle of all this. At just about every moment we've had a lots of people gathered around our booth and it's been really great to be able to demo Kirix Strata™ in person. I'm sure I'll probably be beat by the time I get home, but at the moment, I'm just fired up just to be here. Now, if I had only brought that extra box of business cards along…

During our journey from the MySQL Conference last week to the Web 2.0 Expo this week, we've continued to see a common pattern. There are a tremendous number of back-end business systems and almost all of them seem to rely on the humble Web browser for the front-end.

Now there is a very good reason for this — the Web is a tremendous distribution platform and companies are taking advantage of the same protocols and tools on their intranet that have already propelled the growth of the Internet. And, bottom line, browsers are really, really easy to use.

Data Integration 2.0

As an example of this situation, we met some folks last week from Snaplogic. Their product makes it really simple to integrate data from back-end systems and enables you to access data from anywhere, mash it up and publish it. However, when they were demoing the software, there wasn't particularly much to show off, since a normal browser can't do much with the data except let you “Save As” to your desktop.

Thankfully, this type of front-end work is perfectly suited for a data browser. So, we decided to demo each other's products at the Web 2.0 Expo this week. Snaplogic is able to easily show the value they bring to the table by actually displaying the data integration they can do. We put together a quick dashboard with Snaplogic data and were able to show off all the cool stuff you can do once you've got a data feed to work with. Indeed, this was like the serendipitous combination of chocolate and peanut butter.

The Last Mile

This reminded me of a great metaphor used in a post by Juice Analytics called “The Last Mile of Business Intelligence.” The idea is that businesses have built lots of large back end systems — from data integration and warehousing to BI and reporting systems. But, in the end, it still seems that actual users are left out in the cold. The “last mile” between the heavy machinery and the actual knowledge worker has not been adequately bridged.

So unfortunately, the typical pattern involves working with a back end system via a Web browser and then, ultimately, exporting a CSV file to work with in Excel.

This is not terribly efficient. A data browser enables you to open up those same intranet pages and perform your normal browsing tasks. However, because it's a data analysis tool at heart, you can actually open up data sets and start working with them directly. Then you can do fun things like bookmark tables, build calculations, create reports and write queries and have it refresh the next day with the latest data. Because of this, it plays very nicely in the worlds of of exploration and reporting per Juice's “data value framework.”

Why Incorporate the Web?

In the past we've shown off some of the fun things Kirix Strata™ does — like open up HTML tables natively or showing RSS feeds in a table view — but often we'll get a response like, “that's cool, but what do I do with it?” There is no question that tools like these can be very useful for some tasks (e.g., integrating data from the web with local tables), but they're really just icing on the cake.

The important thing is how tightly the web is integrated with the data analysis tool. This opens up all kinds of possibilities for direct interaction with both the Web infrastructure and the database systems that power business intelligence in the enterprise. Add some Web APIs into the mix, and now you could have access to the contact information in your ACT database, the corporate newsletter contacts in a MySQL system and all the recent sales information from your Salesforce.com account. Explore and integrate at will.

Given that the meme “Enterprise 2.0″ resounds louder each day, we should only expect to see Web infrastructure reaching deeper and deeper into organizations. In the end, maybe part of the BI “last mile” solution is just giving businesses a browser with a bit more intelligence.

Just finished up showing Kirix Strata™ off at the MySQL Conference and Expo in Santa Clara, CA the past couple days.

We had some great crowds, excellent questions and plenty of discussion about how Strata can fit in to a MySQL shop. Besides the hands-on data analysis and querying Strata allows, people seem pretty impressed with how quick and easy it was to generate ad hoc reports.

We also had the chance to meet a lot of great people from various companies — definitely some pretty cool technology out there. One company in particular that we were impressed with was SnapLogic. They're sort of like Yahoo Pipes for the Enterprise — they make data integration really simple. We even had the chance to hook Strata up to some of their data feeds… it was like a match made in heaven.

Well, nothing like doing demos all day for two days in row. We'll try to match that and then some at the Web 2.0 Expo next week from April 23rd through the 25th. If you'll be attending, please make sure to drop by and say “hi”.

We're pleased to announce that Kirix Strata™ is now officially out of beta! A lot of hard work and late nights of coding have been put into this li'l tool and we hope it makes people's lives a little bit easier when it comes to data analysis.

Thanks to everyone who has been involved in the beta process — we'll get those free licenses rolled out to you within the week.

Also, you'll notice a big ol' website redesign too. We want to give a special shout out to Jeff, Benni, Peter and David for the bang up job they did with the design and implementation of the website. And, here's our new overview video, check it out:

(And here's an embeddable YouTube version…)

Now that Strata is out, we're looking forward to adding a lot more content to the site, in terms of more data-centric blog posts, product applications, tips & tricks as well as some open-source projects that we've been working on in the background. Oh yes, we also have a load of extensions that we are planning to release too. So much to do, so little time.

Thanks again to everyone. Please download the full version of Strata and try it out for 30 days free. If you have any questions, feel free to post a message to the forums or shoot us an email.

Great post today by Bret Taylor, seeking a utopian wikipedia for structured data. There are currently various attempts at this type of thing; Freebase, in particular, comes to mind.

But what Bret is talking about is less about the semantic web — where all data everywhere is linked together by certain tags and/or infrastructure — and more just about getting access to really useful chunks of information, like mapping data and white pages. Heck, it'd be great to just get all US area codes with related information in a nice, accessible CSV file.

Hard to know if this sort of thing could ever come to pass, given that there are likely far fewer people who would edit a large financial data table than, say, the latest goings-on in American Idol — however, I'd love to see it happen.

In the meantime, ReadWriteWeb has compiled a bunch of data sources that I thought might prove useful to you as well. Some of this stuff was new to me and I look forward to exploring them. The one thing that the article didn't mention was the great publicdata tag archive still happening on Delicious that we talked about many moons ago. Make sure to check that out too if you are looking for some good public data sets.

After a long, long beta cycle, we're pleased to finally announce a release candidate! Feel free to download it here.

This version takes Beta 9, released last week, and tightens the screws. We've done a lot of bug fixes and added plenty of little things here and there, like cleaning up some GUI items and making our nomenclature consistent. Oh yes — exciting things like determining if a menu item should be “Bookmark Location” or “Bookmark This Location” or if a dialog should be called “New Bookmark” or “Add Bookmark”. Yes, I can tell you're extremely enthused.

We've also tightened up the report writer functionality — the grouping wasn't working very well in Beta 9, but in this version, you can have multiple groups in your report. Also, we added the ability to create reports directly from queries instead of just tables. And, try out Quick Report: highlight a few columns in a table then select New > Report and see what happens.

And, drumroll please…

We finally have a help manual. It is a not-yet-final-draft, so please excuse any grammatical errors. It'll give you all kinds of extra information about doing data analysis using Strata that you probably didn't know existed. Select Help Contents from the Help menu and have a look.

We expect a second release candidate by mid-next week that should have everything tightened down even further… so stay tuned for more good stuff to come.

Please keep the feedback coming, it's really appreciated! Thanks!