Saturday, 19 April 2008

The ProClarity API - Programmatic changes to ProClarity briefing books

We have a client who is using Analysis Services 2000 and ProClarity Analytics Server 6.3 in order to provide balance sheet and financial line information to their business. The system that I designed had to be able to aggregate financial data up a customer hierarchy but be able to aggregate the data into parents as they stood at the time the data was loaded and against the current parent. (i.e. A slowly changing dimension.) However, this dimension consists of over 1.5 million members and there are effectively two versions of this in the database.

Now this on its own is fine-ish, in that with the 3GB switch enabled and some tweaks to the cleaner thread interval the AS solution is stable and performs reasonably well. However, the original requirement did not specify security on these dimensions. As you can guess, there is now a requirement to place security on these dimensions, partially to limit access to the data but more so to provide a way for a given user's reports to open, defaulted to the customers for which they are responsible.

Now trying to add security using a "security cube" isn't really conducive to performance and adding 440 roles to AS, each with a different security context, isn't really good for maintenance or memory. (i.e. Think replica dimensions.) So, what else?

Well, as an approach, I wondered if it is possible to take a template ProClarity briefing book located in a PAS Store, load it and customise it for a given user and write it back to the PAS store. From previous experiences I know that just updating the XML in the PAS database for a given briefing book and pages causes problems down the line, to the point that they may no longer work. So how else? Can you make programmatic changes? I know that there is an API for ProClarity but the documentation is all for VB etc. and not .Net.

In the end I decided to give it a try and started to write an application that would retrieve a result set from a database specifying the template book and pages etc. that are to be customised and the customisations to be made. It also specifies the PAS store into which the book should be ultimately published and security assignments for the new book.

After contacting Microsoft on TechNet for some hints and tips (whose responses are pretty quick and impressive actually) I was on my way. See Hints and Tips for that thread.

The essentials to know, when using C#, are effectively as follows:
  1. In order to have a briefing book loaded within a ProClarity application in memory, use the KProClarityClass.StartupBook method.
  2. Once you have a KProClarityClass object, use the following objects, properties or methods as shown below...

    Application (Object)
    |
    CurrentView (Property)
    |
    PageCtrl (Object)
    |
    KPCommand (Object)
    |
    GetAxisMemberSet (Method)
    |
    KMemberSet

    As an example, use the KProClarityClass's Application property to retrieve a reference to the Application object. Now use the Application object's CurrentView property to retrieve a reference to the the ICurrentView interface etc. etc.
  3. If you make any changes to a KMemberSet, you must use that object's associated KCommand object's execute method in order update all associated entities within the ProClarity object model.
  4. If you copy elements etc. be sure to update the copied element's ID otherwise you will get errors when trying to publish the briefing book. (i.e. You will have two elements with the same ID which understandably is not allowed.) To do this, set the ID for the given element to: "{00000000-0000-0000-0000-000000000000}"
  5. In order to commit changes you have made to a given briefing book page, you have to copy the "page" within the book, using the CurrentView's AddPageToBook method and then delete the original page. Please don't shout at me, I know, that sucks but that's just how it is okay?

In the end I changed my application to run as a custom SSIS task and to be multi-threaded using a blocking-queue in order to hand out template briefing books for updates to consumer threads etc. That all seemed to work although I did experience exceptions when running the task in debug mode. As can be seen by the following thread with Microsoft here, this was related to my code not being aware of the COM aspects of the ProClarity API.

All in all it was quite fun to develop, creating mini-caches for PStore connections etc. and to implement a solution that runs within SSIS. I added in the ability to set properties such as the number of threads to use and the ability to set the table and column names that store the relevant updates to be made.

Using simple test cases it appears to work quite nicely. Reports remain static, such that reports with both grids, totals and graphs etc. retain that layout when the member selections are changed. The task allows the ability to either update members by axis or to simply replace the MDX for the report. (Actually you could replace the MDX for a report and then customise the member selection immediately thereafter if you so wished but why?)

Before it's a finished beta v0.0.0.0.0.0.1 "product" (Oracle style versioning there) the task needs to accommodate pages within nested folders (i.e. currently only handles pages at the root or one level below) and to complete the security aspect. I also need to normalise the table structures that store the updates to be made against the template briefing book etc.

Anyhoo, if anyone out there is interested in using this please add a comment to this blog and I'll see if I can add it to our company's website for download. That naturally assumes anyone actually reads my blog...

Tuesday, 4 December 2007

Oracle 11g on Oracle Linux

Intro
So here's the thing...

I decided to have a more detailed look at Oracle 11g. I like the Oracle relational databases, really I do. They are a mature, stable, scalable database which is feature rich. It can run on Linux, Unix, Windows and even a Mac if you want. However, my focus in this blog is the installation of the database on Linux. Oracle's own in fact...

So Oracle offer a "distro" of Linux. Sounds good as a concept, Oracle offering an OS on which you can install their database and all. However, if you are a newbie to Linux and/or Oracle you gonna need to set aside quite a bit of time for this. (By quite a bit, read: "a lot!")

Now my issue isn't so much the number of settings required during installation, it's a complex piece of software, nor the planning on how it should be or where it should be installed. All that is fine. I understand that. It's just that the actual act of installing Oracle 11g on Oracle Linux should be, well, seamless should it not?

Grumbles
I'm sorry, the fact that I installed Oracle linux with practically all package options selected (didn't select the KDE development ones) yet still didn't get the "oracleasm" packages installed mystifies me. The fact that I didn't get prompted with a question asking "are you intending to use this installation as a platform for another Oracle product" and then providing me with a list, is a little concerning. Some way of being able to configure the installation of the OS based on what I want to do later with other Oracle products seems somewhat natural, yet it is lacking. But what really sends me down a rabbit hole of despair, is the fact that of the 3 "oracleasm" packages required for using Automated Storage Management, one of them is not included in any of the 5 disks of Oracle Enterprise 5 ISO images.

I cannot help but wonder if I am just being picky. It's just that it seems natural to me to include all requirements for a successful installation in the installation software. In order to use ASM I need to have already installed the ASM libraries, yet to do that I need to go to OTN and download the missing package. To make matters worse, the package seems to have been released a long time ago yet the ISO images for Oracle Linux Enterprise 5 have not been updated to include them.

Then, to top it all off, once you actually get the dang thing installed, (i.e. the OS and 11g) there are no installations of start up scripts or anything. Nope. Nada, bupkiss, didley-squat! You want Oracle 11g to start up automatically you darn well get to write them yourself or find someone else's on the web and use that. As it happens, using someone else's is probably natural seeing as by the time you've actually managed to install the software you are probably on to your 2nd or 3rd blog, the writers of which have probably included theirs.

Add to this that if you are using ASM without RAC, you need to include the marking of your RAW disks for oracle ASM in your start up scripts in order to get the ASM service to know about else the Oracle database won't start. (Maybe this is meant to be included in a config file somewhere but I couldn't find any documentation on that. Some blogs talk about using "rawdevices" but that has been deprecated now.)

Finally
The long and short of all of this is that I can fully understand now, why someone would install Microsoft SQL Server for a demo etc. You could have Windows installed, the SQL Server software installed and configured and the demo built whilst you are still faffing about trying to get everything you need for the Oracle installation.

What a shame... In my opinion, for what its worth, the Oracle RDBMS is far better and complete in its offering than SQL Server. I guess I mean it is more mature. But lets face it, money talks, and if at a fraction of the price you can have a database and have it installed dancing on one leg whistling Dixie in the fraction of the time of something else, why wouldn't you?

Links I used:
  1. Install Oracle 11g for RAC
  2. Create RAW devices using LVM