Data and the Web

We’re pleased to announce a long-awaited new upgrade to Kirix Strata today.

Here are a few highlights of note:

1.  Back-end Stuff.  This version includes a TON of work on back-end processing, scripting, CSVs, etc.  So, a lot of the changes will be “invisible” to the average user, but will benefit everyone with speed improvements and a more robust data engine.

2.  Expanded Relationship Filtering.  In the previous version, we had two options for filtering related records — either “leave filter off” or “filter all child records.”  In this version, we’ve added a third option to “mark filtered records” within the context of the entire table.  So, now, when you tile your two related tables horizontally and select Tools > Related Records > Mark Related Records (or select it from the icon dropdown on the toolbar), your related records in the child set will be highlighted in yellow.  In addition, we’ve also added a cursor marker to the parent table so you can track where you are in the child set.

3.  Additional Aggregate Functions.  In addition to the existing aggregate functions (e.g., SUM(), AVG(), etc.), we have included new options for Standard Deviation, STDDEV(), and Variance, VARIANCE().  As with other aggregate functions, you’ll be able to use these in areas such as the query builder, relationships and grouping.

4.  Expanded Table Statistics.  When you select Data > Summarize, you’ll now also get the minimum and maximum field length of each field.

5.  Import Templates.  We’ve added the ability to save import templates (File > Import) to your project, which will save a few steps if you have complex imports.

6.  Bug Fixes.  We’ve been able to knock out loads of small fixes throughout the software.

You can download the latest and greatest here.  If you run into any issues or need help with anything, please just let us know.

We haven’t been doing much regular blogging lately, but we’re hoping this will change in the coming weeks.

In the meantime, we’ve recently done some housekeeping on our website, so if you haven’t visited recently we’d encourage you to do so. We’ve updated many pages with new content, but here are two sections in particular that we’d steer you toward:

• Examples Section.  This is a long overdue section that puts together some quick examples of how Kirix Strata™ can be applied to common data problems.  The section is still a work in progress with more videos still to be produced.  However, we expect what we have now will prove useful to new and old Strata users alike.  Check it out.
• Video Tutorials and Archive.  We’ve done a bunch of different videos and screencasts over the past year or so, but they’ve been they’ve been posted all over our website.  This new section wrangles all of the videos together in one place for posterity.  The feature tutorials, in particular, are worth viewing as they help give a more comprehensive look at how to use specific features in Strata.  Take a look.

So, in a nod to the Matrix, where one cannot be told what it is, but one must see for oneself, we’ve tried to make some high quality video documentation available.  Stay tuned for more to come.  Enjoy!

This is sort of out of the scope of this particular blog, but I thought I’d pass along the news that we just released another open-source library for wxWidgets users. This one is called wxWebConnect and it’s a library for wxWidgets that enables developers to quickly integrate advanced web browser capabilities.

Basically, it wraps up functionality exposed by the Mozilla Foundation’s Gecko engine (XULRunner) into a set of user-friendly classes to: embed browser controls, search web content, print web pages, interact with the DOM, implement custom content handling for different MIME types, issue POST calls using the current browser state, etc. Notably, with this library you can also embed all of your favorite Firefox browser plug-ins into your application. We’ve also gone out of our way to make sure that getting a browser control up and running in your application is as easy as possible.

More information can be found at the wxWebConnect project page. Also, feel free to view some screenshots and a short video demonstration too. If you’re a wxWidgets developer, give it a whirl and let us know what you think.

We’re pleased to announce that we just released a new upgrade to Kirix Strata, version 4.3! Kudos to our developers for adding a lot of nice features and bug fixes.  The full list of notes to this release is below the jump, but here are a few of the bigger changes:

External Database Connectivity

We’ve really improved the way that Strata works with external databases by optimizing our pass-through queries for databases like Oracle, SQL Server and MySQL.  In addition, queries in the query builder that reference external database tables pass the query through to the external database, significantly increasing the speed of queries on external databases.  Furthermore, you can now edit individual cells in Strata and have them update in your external database table.  This is very welcome news to folks that want to use Strata as a front-end to their external database tables.

UPDATE (04/30/2009):  Just a quick note of clarification, on the “read” side, you can work with external databases for things like sorting, filtering, marks, calculated fields, grouping and copying.  On the “write” side, we currently only have cell editing available, but will work on adding other features in the future such as append (i.e., insert record), delete, update, and some modify structure operations.  If you need these additional “write” features, please send us a note to let us know how you would plan on using them to help us prioritize our development efforts. Thanks!

Improved SQL Support

We added a console panel to allow direct querying of internal and external databases with SQL commands, as well as to provide feedback for database operations and scripts.  You can learn more here.

EBCDIC Conversion

Strata now handles EBCDIC.  We haven’t added copybook support just yet, but you can either manually set your breaks using the text-import or create scripts to convert the EBCDIC file to ASCII format.  You can learn more here.

Fixed Length and Delimited Table Export

We’ve also added Fixed-length export (this also works when using File > Save As External).  In addition, we’ve expanded the text-delimited export so that you can specify your own delimiters, such as pipe-delimited and semi-colon delimited. You can learn more about the new text-delimited functionality here.

Handling Tablenames & Fieldnames with Spaces

One of our most common support questions relates to spaces in a fieldname (like “my field” instead of “my_field”).  We’ve now solved this issue by allowing spaces to be used by enclosing the name in brackets.  So, for example, these are now all valid expressions:

[Field  1] * [Field  2]
Field1 * [Field  2]
[Table 1].[Field 1]*[Table 2].Field2

You can learn more here.

Much Much More…

There are plenty of other upgrades like project handling, new keyboard shortcuts, auto-fill group and sort dialogs, new script classes, etc.  You can check out all the changes below the jump.

Please download the latest Strata (or just click “Check for Updates” in the Help menu), give it a whirl and let us know what you think!

(more…)

As a follow up to my previous post on e-government, just wanted to let those who are interested know that there’s a free conference offered next week that will get much more in-depth about the initiatives for changing the way government uses and disburses information.  The conference will also have a particular emphasis on using semantic technologies.

Here are the details:

From E-Gov to Connected Governance: the Role of Cloud Computing, Web 2.0 and Web 3.0 Semantic Technologies

Tuesday, February 17, 2009.

Morning session: 8:30 am EST to 12:00 noon. Afternoon session: 1:00 pm EST to 4:00 pm EST.

Synopsis:  “We have a new administration that values transparency, citizen participation, collaboration, information sharing, and internet technology… The purpose of this conference is to operationalize this vision, demonstrate the kinds of changes that are coming to next stage web-based systems in government, and to map the role of  information and communication technologies (specifically, cloud computing, Web 2.0, and Web 3.0 semantic technologies) in the evolution of government information systems from e-gov (silos with web front ends) to connected governance (e.g. distributed social computing environments for collaborative work, information sharing, knowledge management, and participatory decision-making.)”

Webcast sign-up here (or, if you are in Washington DC area, you could attend in person)

Further information about the conference can be found here.

We’re happy to announce another new Strata upgrade today.  If you’ve got a version of Kirix Strata 4.x on your system, this is a free minor upgrade, so download it now.

This release contains a ton of various bug fixes and adds a few notable features:

Saved Indexes

Indexes are fundamental to the inner workings of a database.  Strata has always saved indexes per session for things like sorts and relationships; however now you can save them from session to session so they don’t need to be recreated.  You can save indexes by choosing Edit Indexes from the data menu.  Once saved, you can just right-click on any field header and select Sort Orders to access your saved sorts instantly.

Saved Column Views

When performing your analysis, it is almost inevitable that, at some point, you’ll change the way your columns are shown.  Some will be rearranged, others will be hidden.  However, particularly with data sets that contain many fields, it is often useful to be able to save column views to access at different times.  You can now save column views in Strata by selecting Data > Columns > Edit Column Views.  Once you save your views, you can easily access them by right-clicking on any field header and selecting Column Views.

Function Help

We’ve made it a little bit easier to get help with various functions when you use the formula/expression builder.  If you hover over any function in the list, you’ll see a tool tip with the function syntax and short description.  If you click on a function, you’ll see the syntax appear on the bottom left of the formula builder with a hyperlink to further information.  Functions are such an integral part of data analysis that we hope this extra little bit of help will make things faster.

Scripting/Extensions

Overall, we’ve fixed lots of different bugs and have continued to improve the strength and breadth of our scripting language.  Because of some scripting tweaks, some of the previous extensions you downloaded may not be visible in your menu, as expected.  If this is the case, we’ve updated all the extensions with the new syntax — so just go ahead and re-install the extension and you’ll be ready to roll.

As always, if you have any recommendations or see any bugs, please let us know — it is extremely helpful to have this type of feedback.

Just a quick note that we released a minor update to Strata today, which includes some bug fixes but mostly added some important bits and bobs to our scripting API. Besides a bunch of scripting fixes, we added a timer class and asynchronous events to HttpRequest and FileTransfer classes. You’ll see these in action soon for some of the extensions we have in our own queue. Or, feel free to try them out yourself.

In addition, please note that if you are using the Benford’s Analysis extension we mentioned in the previous post, it too has been upgraded to deal with a pesky field naming bug. It is backwards compatible with 4.1, but the old extension will not work with 4.1.1. You can install the upgraded extension from here.

Little ads: dvd movie downloads

 We’re happy to announce the release of Kirix Strata 4.1, which is a maintenance upgrade that adds some new functionality and also fixes some problems.  Here are some of the new and improved items in this version:

Reports

• Added the ability to create formulas within reports.  To add a formula to a cell in the report design view, just begin the expression with an equal (=) sign.  These formulas allow you to use all of the functions that you normally use in calculated fields.
• Added a right-click option to insert both common, pre-built formulas into cells as well as custom formulas.  Some pre-built formulas include the current date, page numbers and and page count.
• Improved the usability of the report design view and fixed some drawing problems.

Connectivity

• Added the ability to access database views directly from database connections.  In the previous version, when you connected to some of the external databases like Oracle or SQL Server, you only were able to access the tables; now, you can also access the views in your database.
• Added the ability to open additional data tables, such as TSV files, directly from the web. Data tables that are opened directly from web tables now use the MIME type to load properly rather than relying exclusively on the file extension.

User Interface Enhancements and Fixes

• Added a German translation for Strata’s menus, dialogs, and other parts of the interface.  However, please note that the documentation remains in English only.
• Fixed a problem where the software would crash if the first mark color was changed from the default and then an additional mark was created.
• Fixed a problem that prevented new projects from being created on Linux.
• Added an option to download extensions, instead of just install them.
• Improved the structure checking for tables and queries.

Scripting

• Added additional script functions for integrating scripts with the main Strata application, interacting with a web page’s document object model (DOM) and passing post data in HTTP requests.
• Added improvements to considerably increase script load times.
• Added additional SQL functionality for connecting to different databases and converting from numeric and date values to character values.

This upgrade is free to anyone with Strata 4.0, so please download the new version, or simply, select Help > Check for Updates inside Strata.  Then, let us know what you think!

In addition to today’s announcement about the Extensions section, we’ve also released an equally important new part of our website — the Kirix Strata™ Blog.

If you use Kirix Strata, we would recommend you subscribe to the Strata Blog feed (or get a subscription via email).  This will be the place where we  post examples, tips & tricks, case studies and interesting links.  There are lots of things that Strata can do to make your data tasks more efficient or let you discover new things within your data; you just need to know how to use the tools in the toolkit.

If you have data questions or would like us to demonstrate a particular concept, please let us know and we may be able to create a Strata Blog post for you and let everyone join in on the education.

Also, while we’re talking about feed subscriptions, the other feed that may interest you is our Extensions feed (again, this can be subscribed to via email).  This feed will alert you whenever a new extension has been posted to the Extension Library so you can try out the various things that interest you most.

As with any new blogs, we are obviously starting with a humble first post, but we plan to expand rapidly from here.  And, if you have any feedback on these two new sections, please let us know how we can best serve you.  Enjoy!

We’re really excited to announce our Kirix Strata™ Extensions section along with a load of resources for developers.

As with other browsers (like Firefox, which is making some news of its own today), Kirix Strata is extensible and supports a plug-in architecture.  It uses the JavaScript syntax, so any developers who are familiar with Javascript should find creating Strata scripts pretty easy.

Unlike other browsers though, Strata packs a full database engine for the journey. Combine this power with the ability to access stuff on the web (web content, APIs, DOM manipulation) as well as local files, and you’ve got yourself a highly customizable rich internet application for data-centric tasks.

We’ve been scripting a lot for client projects and it has pleasantly surprised us how much one can do with Strata’s engine.  We’ve also been creating a bunch of extensions in-house that we’ll be rolling out in the coming weeks for everyone to use.

Here’s a full list of the stuff we’ve added to the website today:

Extension Library

The library is the place where we’ll be listing all new extensions.  We’ll be rolling these out as we create them, but we’d also be really happy to publish any extensions developed by the community that may be useful to a wide range of people.  Got an extension to share? Please submit it and we’ll post it.

Extension Wizard

The Extension Wizard makes some of the mundane tasks of creating scripts and extensions a little less painful.  There are three things it offers:

1. Extension Packaging:  Create the appropriate “packaging” for an extension.  Just write your script and let the wizard package it up for you to distribute.
2. Script Templates and Components: This area provides a number of pre-packaged scripts that you can use in your own development.  It has scripts for such things as form controls, form layouts, database/SQL examples and API examples (e.g., FTP requests and an RSS feed parser)
3. Sample Applications:  You can also create variations of a full application which is helpful when you want to take an already-built extension, open it up and see what makes it tick.

Build Your Own Extension

The build-your-own page provides a high-level view of creating an extension for Strata.

Developer Resources

With the Developer Resources section, we’ve finally put some meat around the skeleton API documentation that we previously made available on our website.  This section provides an overview of working with scripts in Strata as well as information about the syntax and API.

Submit an Extension

Got an extension that you’d like to share with the world?  Submit it here and we’ll post it to the library.

Kirix Strata does a lot of great stuff out of the box for working with and reporting on data.  But scripting and extensions offer power users an opportunity to develop customized applications for themselves and their co-workers.  If you are a developer, we hope you dig into the documentation and find it valuable.  If you aren’t a developer, we just hope that the extensions library will  prove useful to you over time.

Lastly, please let us know if you have any questions about scripting or extensions and we’ll be happy to help!