Data and the Web

Hello everyone,

We're happy to announce that Kirix Strata Beta 8 is now available for download!

There's been lots of good work on Strata over the past month or so, with much of it focused on bug fixes and other tweaks for a better overall user experience. That said, here are a list of some of the bigger things we've tackled for Beta 8:

Data Conversion

We continue to try and make opening, saving, connecting to and integrating data as seamless as possible. We've taken another good step in this direction by enabling “Save As” to work between external database systems and external folders. So, for example, now you can do things like connect to a MySQL database and drag a table from there into an Oracle database system. This basically lets you copy tables from one system (or local folder) to another. And, of course, you can also do this same type of copying operation with data files already saved to your Strata project.

User Interface Enhancements

• We've done a lot more work on the Bookmarks toolbar. We're also delighted to finally have favicons in, which makes things feel slightly more “normal” when browsing and using bookmarks.
• The Project Manager has been given a very nice face lift and is far more usable than it was in Beta 7.

Report Writer

We've optimized the layout engine so that reports created with large data sets are now generated SIGNIFICANTLY faster.

Scripting

We've added the following functionality:

• Support for FTP and SFTP to push data back and forth. We're working on some example extensions that will make great use of this functionality.
• A new, improved ListView control.
• A HostPreferences class for storing and retrieving application and extension preferences.
• A HostApp.createPane() and HostApp.createDocument() so that forms can be added to the application as docked panels or documents.

As always, please take the new version for a spin and let us know what you think. Bug reports and all other feedback are more than welcome. Thanks!

(NOTE: See screencast video below for a quick look at some of the new features!)

Hope everyone had a lovely holiday season!

We're happy to report that our developers provided lots of shiny new toys in our Strata stocking over this past month, including further work on Data Links, the inclusion of a “Quick Filter” mechanism and the introduction of our new report writer. Please feel free to download Strata Beta 7 and let us know what you think!

Here's more information on what's new in this latest version:

Data Links

The ability to bookmark data files is coming into its own. We've got things working pretty well on CSV and RSS files at the moment, with some more work still to do on HTML tables. Here's a general synopsis:

1. Open a CSV or RSS table from the web.
2. Perform your own analysis, using calculated fields or marks.
3. Save the data URL as a simple bookmark.
4. Click the Refresh icon or open up the bookmark in the future. Your data (and your calculations) will refresh based upon the new or updated data on the server.

We've been finding this quite useful internally, particularly in relation to analyzing our web log data. Check out the screencast below for further info.

Report Writer

With Beta 7, we are also introducing our new report writer.

You can create your report in a design view (similar to a template) and then toggle to a layout view for a preview of what you'll see when you print. As a bonus, the layout view enables you to manipulate and format your data directly, instead of being bound to a “print preview” mode.

Another cool thing is that, besides creating reports from data in your project, you can also create reports directly from external data, such as local CSVs or MySQL tables. (First go to File > Create Connection, then you can select it as your source data in the report writer). Check out the screencast below for a quick demo of the report writer in action.

Please note that there are a few known bugs with Report Writer in Beta 7. These include:

• When using groups, the first group does not display properly.
• The layout view can be extremely slow when using large files. Now that we've got some big features in, optimizations will soon follow.
• Items in the Report Header in the design view do not display properly on the top of the page.

Other Enhancements

Here are some of the other improvements that have been implemented:

• Quick filter allows tables to be filtered really easily (see screencast below for a quick demonstration).
• Quick import for MS Access and Kirix Package file via the File > Open command instead of File > Import.
• Support for CSV files with Unicode character sets.
• CSV auto-sensing determines the field delimiter so lots of different delimited files are parsed and opened automatically (e.g. comma, tab, semi-colon, colon, pipe, tilde).
• A bunch of scripting additions, including functions to access a database table list and table structure information. We've also added functions to encrypt/decrypt strings.
• Automatic plugin detection (Strata now doesn't need to reinstall programs like Flash plug-ins if you have already downloaded them for other browsers).
• Streamlined extension installation and uninstallation.
• A new “loading” icon that appears on tabs while web pages are being downloaded.

Please check out this screencast, which provides an overview of Data Links, Quick Filter and Report Writer:

(And here's an embeddable YouTube version…)

NOTE: For those interested, here is the Yahoo URL used in this screencast. Check out Gummy Stuff's extremely useful Yahoo Stock Ticker CSV API site for further information.

Thanks for downloading it and giving it a spin. Please let us know if you run into any bugs or need help with anything!

We've got the new Strata beta 6 ready and available for download so you can play with the new features before the legendary tryptophan coma kicks in tomorrow (at least for our U.S. readers).

We've got a couple big features that we've added to this beta:

Data Bookmarks

Ever since we've seen web data transformed into nice, usable tables, we've longed for the ability to a) link to the data and b) refresh it.

Today, we've taken a big step in this direction. You can now link directly to data tables on the web. The two formats we support in this version are RSS and CSV, but more will follow in the coming weeks.

So, for a quick example, copy the following URL and paste it into Strata's URL bar:

http://finance.google.com/finance/historical?q=NASDAQ:AAPL

Google Finance provides historical stock information, in this case for Apple. This website also provides a “Download to Spreadsheet” link that, if clicked in Strata, will open up the CSV file in a table. In previous beta versions, you couldn't link to these tables directly. However, in this version, you'll see that it is just a regular ol' URL waiting to be bookmarked:

http://finance.google.com/finance/historical?q=NASDAQ:AAPL&output=csv

And, with regular ol' URLs, you can do things like open them up tomorrow and view updated data. Or, you can share the links with friends and co-workers. You can also do things like create a calculated field that will automagically recalculate tomorrow when the data changes.

We've got these data links up and running for RSS feeds too, say:

http://www.digg.com/rss/index.xml

However, we don't have the saved calculated functionality in for RSS just yet. Also, right now the “refresh” button in the toolbar isn't hooked up to these links, but it will be soon.

Because URLs are such a simple concept that everyone understands, we envision that this type of thing could be quite useful — particularly in a corporate environment. Instead of creating a daily CSV/TXT extract that must constantly be downloaded and loaded into Excel/Access on the desktop, an analyst could just create a data link in Strata and click the bookmark each time they wish to access the data.

Got other ideas for how you might use them? Please let us know what you think.

Automatic Update Notices

We've added a feature that will provide a notification of a new beta version without you ever needing to leave the comforts of your own Strata. Hopefully this will more efficiently keep everyone up to date with the latest and greatest. This doesn't mean you should unsubscribe to this blog though.

Other Features/Fixes

• The Bookmarks Toolbar continues to get enhanced. For instance, now you can drag links from the URL bar directly to the link bar, just like you'd normally do in Firefox or Internet Explorer.
• Downloads from the web are now hooked up to the Job Manager.
• If jobs are running, you can now stop them with the Stop button (or will cause the job manager to open if there are multiple jobs running).

We really appreciate all the comments and feedback. Please keep it up, it's fantastic!

It's been a while since the last beta download was released, but the time has been well spent cleaning up bugs and creating some fairly time-intensive features. There are two big changes, both related to the GUI…

All Your Links Are Belong To Us

We've been working hard on the concepts of “links” in Strata. In a normal web browser, a link is merely a pointer to a URL. In a data browser, you also want to be able to point to things like HTML tables, Oracle database tables, a CSV file on a network drive and, of course, traditional URLs. The holy grail, of course, is when you can also start “refreshing” these data links so that you always have the latest data to work with.

We don't have the refreshable links just yet, but we do have the very first incarnation of our Links/Bookmarks Toolbar. It's not perfect yet, but we'll be making it even better in the next week.

A few of the bigger known issues:

• You cannot yet drag a link from the URL bar directly to the Link Toolbar. For now, to create a link, you need to use the “Star” icon next to the URL. However, note that you can drag items from your Project Tree into the Link Toolbar to create shortcuts to tables, queries, scripts, etc.
• You can create a Folder on the Link Toolbar by right-clicking and selecting “New Folder.” You can then drag individual links into the Folder — but you need to drag it into the white “drop” area for this to work.
• The right-click menus have been disabled within the folders, which means you can't delete links, move links or create subfolders.

We're glad to finally have this feature in the software. We hope that, despite the rough edges, you'll find them as useful as we do.

Virtual Mr. Potato Head

So you have this thing called a data browser that does both a lot of “data” stuff and a lot of “web” stuff. Unfortunately, when you have all this functionality that traverses these different genres, you have a lot of classic interface elements that are just askin' to be squeezed in. But, alas and alack, if you just combine everything, you end up with a large, flaming ball of complexity. A lot of very worthy interface candidates end up fighting for a limited supply of screen real estate.

But design limitations aren't necessarily a bad thing. In the end, we've tried to make some choices that provided the best bang for the buck. However, we're still tweaking things… so if you have some ideas for improvement, speak now or forever hold your peace.

Here are the main areas we changed around:

Navigation Toolbar

The top toolbar is now used primarily for navigation. Your standard browser controls are on the left, you can search with the find box on the right and you can switch your views by toggling the icon on the far right.

File/Links Toolbar Combo

The second toolbar is mainly used for links and bookmarks, as discussed above. However, we also added in the most common elements for working with files and tables, like you'd encounter in a spreadsheet. This includes the new/open/save icons along with buttons for standard data operations (sorting, create calculated fields, etc).

Status Bar

We've redesigned this to provide a bit more space so we could include easy access to the various “panels” in the software. This area makes it easy to toggle on/off the project tree, the column list, marks, etc.

We're still rounding things out a bit; here are a few of the known issues:

• Please avoid turning toolbars on/off in the View menu or you'll get a very bad display bug which requires a bit of monkeying around to undo.
• In the status bar, we've added a “format” panel; however, this is just a placeholder at the moment. The wiring hasn't been hooked up just yet.
• In the status bar, the toggling doesn't always work correctly.
• Some of the icons need to be re-worked for clarity.

So, please go ahead and download the new version and let us know what you think via our forums or feedback form. It's really, really appreciated. Thanks!

It's been a few weeks since our last beta release, but the time has been spent well. We've focused on fixing a wide-ranging number of annoying bugs; the new Beta 4 is available for download now.

Here are a sampling of features and bug fixes that have made their way into this latest version:

• ATOM Support as suggested in this forum post
• A bunch of RSS compatibility fixes
• SQL compatibility updates
• Scripting now includes a DOM API subset
• Lots of keyboard navigation work
• The structure editor view now allows field reordering
• Strata can now handle websites with unsigned certificates
• mailto: links handled properly as reported in this forum post
• Tons of other bug fixes, including the notorious appmain shutdown zombie

Keep the bug reports and suggestions a-comin'. Thanks!

Hello all, we're quite happy to announce that we've got our third beta version up and available for download. Here are the major features/bug fixes that made it into this version:

• Vastly improved Oracle connectivity.
• A bunch of MySQL connectivity bug fixes.
• Import wizard refinements, including a better interface for choosing what type of file you wish to import.
• Loads of other bug fixes based on user feedback.

Also, we've done some work in the area of scripting and extensions that should be highlighted:

• Extensions can now be enabled/disabled from the extension manager.
• With scripting, you are now able to load and execute function calls to DLLs and shared libraries — a special shout out to kapex01 for suggesting this feature, we think it adds a lot of really cool possibilities for using Strata as a platform for combining the best of desktop with the best of the web.
• Also now with Strata's scripting, you can directly access window handles for forms and controls.

Thanks again to everyone for posting bugs and feature suggestions in the forums and via our bug feedback form, it is a tremendous help!

Hi everyone, we've now got a new beta iteration up and available for download.

Strata Beta 2 includes the following new features and bug fixes:

• Added proxy configuration support (see screenshot above) as requested here. To adjust your settings, go to Tools>Options>Internet Tab.
• Fixed the overly-eager numeric auto-sensing for things such as ip addresses (12.233.132.33) and English Premier League scores (1-2) first reported here.
• Fixed the web page source view issue as reported here. Note that you can now toggle between views by either using the “View” menu, by clicking on the “Toggle View” icon from the toolbar or via the right-click menu on web pages.
• Fixed the Ubuntu installation issues as reported in many places.
• Fixed the MySQL DateTime issue reported here.
• Also fixed the MySQL off-by-one issue as reported here.
• Fixed the column break issue reported here.
• Fixed a whole bunch of additional nickel and dime issues such as adding new hot keys, fixing some menu issues, and cleaning up some scripting bugs.
• We also have updated our documentation related to Strata's scripting. Still a lot more to go, but it's something we'll be building on in the coming weeks.

We've also upgraded our build process so hopefully we'll be able to turnaround new beta iterations more frequently from here on out. Thanks to everyone who has contributed to this beta effort (and, as an aside, have earned themselves a free license when Strata is released), either via the forum, the bug report form or via support emails. Please keep ‘em coming!

Thanks for all the bug reports this week; we're working hard to sort them out.

We're hoping to have a new beta for everyone early next week. In addition to a lot of nickel and dime fixes, we'll definitely be adding a configuration page for proxy settings, our most requested feature.

Have a good weekend!

Yesterday, Kirix Strata™ received a nice write up by The Register. Unfortunately we had a little rough sailing in the morning with the ensuing web traffic. We apologize to anyone who had to suffer through worse-than-dial-up speeds while downloading the beta. The problem was fixed and so hopefully it won't occur again.

The article hinted at the origins of Strata and I thought it may be useful to fill this story out a little bit more. Thankfully, this prequel does not involve midichlorians.

We introduced Strata at LinuxWorld a couple years ago as a “dynamic database” — sort of a cross between a desktop database and a spreadsheet — that made it really easy to use, manipulate and analyze structured data. In addition to its ease of use, Strata also had tremendous data capacity and speed, bringing the difficult world of databases a step closer to those who would otherwise shiver at the sight of SQL. These traits actually helped it win the LinuxWorld “Best in Show” award for desktop/productivity/business applications.

Unfortunately, there were a couple issues that limited its mass appeal. The first issue was connectivity: users needed to import all their data into the project. This isn't a problem if the data is static, historical data, but it becomes a bigger problem if the data requires regular updating. The second problem was repeatability: users couldn't easily replicate logic without performing a set of manual steps. So, unfortunately, for any type of repeated analysis, use on a daily basis could become burdensome.

These two areas became the primary focus of the new and improved Strata. We wanted 1) to enable people to work with data outside of a Strata “project” and 2) to provide a way to code their logic into scripts that could be run on a regular basis. For the former, we added the ability to open up files and manipulate them directly, like a CSV file on your desktop or a MySQL table on a server. For the latter, we implemented a scripting language (ECMAScript) with both a database and interface API, enabling developers to create a repeatable process (in embedded SQL) that could easily be deployed as an extension.

But then, as we looked at data accessibility and how to connect to various data sources, we started thinking about the web as a database. Although the web contains large amounts of information in HTML, quite a bit of information can be interpreted in a structured way with just a little bit of work. And with other data available as CSVs, RSS feeds, or through APIs, we thought it would also be useful to allow users to access some of these web-based data resources more directly.

That meant we needed to embed a browser, and after investigating several options, we settled on Mozilla's Gecko layout engine. Of course, the real trick was not just to let people browse web pages, but to let them interact with the content in a more data-oriented manner — a “trick” we're still exploring, implementing and refining. So, in many ways, Strata is a bit more of a “Data Interactor” than a “Data Browser.” But, I suppose, the former doesn't roll off the tongue as nicely…

So, in the end, this beta version of Strata builds really builds on a history of database power and analytics. And now we've got a chance to see what happens when we apply these things to the web too.

Well, it took a lot more blood, sweat and tears than we expected, but we're really excited to announce our first public beta release of Kirix Strata™, the data browser.

And what, pray tell, is a “data browser”?

Well, Strata is a specialty browser that lets you access and manipulate data from pretty much anywhere on the web. For instance, Strata will let you grab HTML tables or RSS Feeds or even open up CSV files directly from a URL (wow, that's a lot of acronyms).

Then when you've got the data in a table, you can do all sorts of ad hoc analysis. You can create calculations or sort and filter or create queries and reports — similar to the kinds of things you might do with a desktop database or a spreadsheet. In addition to web data, you can still work with data from your desktop or in a database system like Oracle or MySQL Enterprise.

And for those more technically-inclined, Strata also includes an implementation of ECMAScript — so anyone familiar with Javascript should feel right at home. The nice thing about the scripting is that it also includes bindings for SQL and HTTP — which can make for a lot of fun when connecting to Web APIs, creating “desktop mashups” or building extensions. And to boot, it runs on both Windows and Linux (at this moment, only Ubuntu is supported officially).

We also just want to give a quick shout out to the excellent folks at wxWidgets (we use their GUI library) and Mozilla (Strata incorporates the Gecko engine) — without which, Strata would only be a mere twinkle in our eye.

So, without further ado, check out the Kirix Strata introduction video:

(And here's an embeddable YouTube version…)

and then

Download and try out the data browser for yourself

We hope you enjoy it!