Claesson.org

My last post about Solaris 11 triggered an interesting discussion with UX-admin about IPS and the lack of pre-/post-scripts for installation/removal.

Assuming that Solaris 11 will be based on IPS just like OpenSolaris, it is certain that sysadmins will have to change a lot in how they create and manage packages, because IPS is fundamentally different from the good old System V packaging system.

So let's explore the lack of scripting hooks in IPS and see if we can find some ways of working around them:

The Problem

One of the fundamental differences between IPS and traditional System V packaging in versions of Solaris <= 10 is that in the old days, you could include pre-install, post-install, pre-remove and post-remove scripts along with any files into your package and the package installation/removal process would automatically start those scripts at the right time during the installation/removal process.

But IPS got rid of all scripts. The reasoning here is that scripting around the installation process introduced more evils than goods, in particular when the setting was more complicated than a plain vanilla standard installation. For a full discussion, check out Stephen Hahn's pkg(5): a no scripting zone article.

In essence, all IPS supports is copying files into the right directories at installation times, and removing those files at de-installation times, plus some simple IPS actions such as creating users/groups or (through the magic of setting special variables) starting/stopping SMF services as part of the package installation process. Check out this simple IPS creation example to get a feel for it.

So if (like UX-admin, whose real name I'd like to know) you really want to do something other than just copying files as part of the package installation process, you won't be happy with IPS alone.

The Background

As Stephen Hahn explained in the essay mentioned above, there's a conscious trade-off in the design of IPS that tries to separate the process of installation from the rest of the tasks that traditionally were part of package management.

Let's check the full package life-cycle in a traditional System-V package install/removal process:

1. The admin starts the package installation process through pkgadd(1M).
2. The pre-install script is executed, which carries out any useful tasks prior to package installation.
3. The files that make up the package are installed.
4. The post-install script is executed, performing any configuration and other post-install tasks.
5. At some point, the admin decides she no longer want the package and starts the package removal process through pkgrm(1M)
6. The pre-remove script is executed, cleaning up anything that needs to be cleaned up prior to the removal of any files.
7. The files that make up the package are removed.
8. The post-remove script carries out any last cleaning actions that had to wait for after file removal.
9. Done.

A full discussion of these steps and how they're implemented in traditional System V packaging fashion in Solaris can be found in the Application Packaging Developer's Guide, particularly in the Procedure Scripts chapter.

(The whole thing is actually much more powerful and complex, but for the sake of brevity, lets concentrate here on procedure scripts alone.)

IMHO, the real issue here is that steps 3 and 7 above are very easy to handle (because they just involve copying/removing of files), while all the other steps (executing the scripts) can be arbitrarily complex, because the system has no control over what those scripts actually do, nor can the packaging infrastructure always trust the scripts to always behave correctly in all possible situations (such as complex network installations, zones, security constraints, etc.).

So the easy solution is: Only allow the installation of files, leave the rest to the admin.

Effectively, this separates the task of Package installation/removal from the task of Package lifecycle management. IPS takes care of the former, the latter is left as an exercise for and as a responsibility of the admin.

The Simple Solution

If all you want is to get your pre/post install/remove scripts back, then you could wrap your old scripts into two installer/uninstaller scripts, deliver these as an IPS package and ask the admin to execute them:

1. The admin installs an IPS package, which copies the install/remove scripts into some staging area.
2. The admin starts the install script, which goes through the same pre-install, copy files, post-install sequence outlined above.
3. At some point, she decides to deinstall everything, so...
4. The admin starts the uninstall script, which goes through the same pre-remove, delete files, post-remove sequence as with traditional package management.
5. The admin removes the IPS package, which removes the install/remove scripts as well.
6. Done.

While this should certainly work, this is not really elegant, because it relies too much on the sysadmin to think (and know) of the install/uninstall scripts. There has to be a better way.

The Comfortable Solution: SMF to the Rescue

And there is. One powerful feature of IPS is the ability to install a new SMF service (by copying its method script and manifest into the right directories), then activate it (by setting a magic variable like this: $ pkgsend add set restart_fmri=svc:/system/manifest-import:default xml_manifest).

We can now use this SMF service to watch over the package lifecycle and to execute the missing scripts at the right time. I'd like to call this SMF service a "shepherd" service, because it watches and takes care over our package in a loving way during its whole lifecycle.

Using SMF to manage the package lifecycle could look like this:

1. Our admin now installs a new package through IPS.
2. The IPS package installs the files to be installed into a staging area, then it installs a shepherd SMF service and activates it.
3. The shepherd service notices that the actual package it's supposed to take care of isn't present yet. It then:
   1. Executes the pre-install script, just like in the old System V days,
   2. Copies the files from the staging area into the right locations,
   3. Executes the post-install script, just like in the old System V days.
   4. Writes the result and the success of its operations into some log file.
4. Now the package is installed, in exactly the same way as if it was installed using the old, traditional System V method.
5. After some time, our admin decides to get rid of the package, so she starts the package removal process by deinstalling the previously installed package through IPS:
6. IPS stops the SMF shepherd service. This triggers its "shutdown" method, which:
   1. Executes the pre-remove script, just like in the old System V days,
   2. Removes the files from the right locations,
   3. Executes the post-remove script, just like in the old System V days.
   4. Logs any errors or progress to some log file.
7. Now, the only job left to do for IPS is to remove the shepherd SMF service as well as the files from the staging area, which it can do automatically.

Notice that the shepherd SMF service now assumes responsibility for installing the "inner package" by making sure it's only executed once and that the execution completes successfully. Plenty of error potential here, but I guess this is the cost of making the install process script-based.

In theory, this should work and I see no reason why it shouldn't. Hopefully, this is a valid workaround for anybody who truely misses the good old pre/post install/remove scripts.

Of course, there's still potential for making this more elegant or sophisticated, for example, instead of staging the actual files to install, the shepherd could use a standard IPS "inner package" instead and issue the right commands to install/remove it.

Some care should be taken to avoid the whole thing to break by the sysadmin accidentally deactivating the shepherd service. This could be done by adding some code that detects the reason for it being called with the shutdown method. This may need some tweaking, but I doubt it would be difficult to do.

Conclusion

The limitations of IPS in terms of scripting have their reasons and IMHO, they're valid. On the other hand, there are other valid reasons for using scripts during the install process as well, for certain situations.

A potential solution is to use the combination of IPS and SMF to carry out both tasks of managing package files and managing the package lifecycle, including pre/post install/remove scripts. One example is outlined above, and I'm sure there are other possibilities.

The point here is: The old installation mechanism has introduced a lot of problems (particularly with patching, as Ben pointed out) and the new mechanism aims at removing complexity by reducing the package installation process to only copying/removing files (and some minor other things). This leads to simplicity, but still allows for arbitrary complexity (including getting your scripts back) through leveraging other features of Solaris, mainly SMF.

IPS is still in the beginning, who knows how it will look like when Solaris 11 is released? But I hope to have highlighted one way for package builders to have their package and script it, too.

What are your thoughts around IPS and scripting? Did anyone try to work around the scripting limitation? How? Do you find the approach above feasible?

Leave a comment!

var flattr_uid = '26528'; var flattr_tle = 'How To Add Pre-/Post-Scripts to IPS Packages'; var flattr_dsc = 'My <a href="http://constantin.glez.de/blog/2010/08/oracle-solaris-11-future">last post about Solaris 11</a> triggered <a href="http://constantin.glez.de/blog/2010/08/oracle-solaris-11-future#comment-69576619">an interesting discussion with UX-admin about IPS and the lack of pre-/post-scripts for installation/removal</a>.Assuming that Solaris 11 will be based on <a href="http://hub.opensolaris.org/bin/view/Project+pkg/">IPS</a> just like OpenSolaris, it is certain that sysadmins will have to change a lot in how they create and manage packages, because IPS is fundamentally different from the good old System V packaging system.So let's explore the lack of scripting hooks in IPS and see if we can find some ways of working around them:'; var flattr_tag = 'hack,ips,opensolaris,Oracle Solaris,packaging,smf,solaris 11,workaround'; var flattr_cat = 'text'; var flattr_url = 'http://constantin.glez.de/blog/2010/08/how-add-pre-post-scripts-ips-packages'; var flattr_lng = 'en_GB'

A summary of the latest updates to the System Admin and Developer Community of OTN, plus posts to the OTN Garage on Facebook. - Rick

picture courtesy of a Peruvian compadre of mine.

Here's a summary of the latest updates to the System Admin and Developer Community of OTN, plus posts to the OTN Garage on Facebook. Give me a shout if you have any other news to share.

New Technical Articles

Using Oracle Solaris 10 to Overcome Security Challenges

How to use the security features of Solaris to combat intrusion and meet requirements for security, privacy, and worldwide internal auditing standards. Privileges, hardening, networking, virtualized environments, certifications, and more.

Increasing Application Availability with Oracle VM Server for SPARC: An Oracle Database Example

How to use the warm migration feature of the technology previously known as LDOMS to increase the availability of an Oracle Database 10g Release 2 single-instance database.

Deploying Web 2.0 Applications on Oracle Servers

Best way to deploy Web 2.0 applications on Oracle Sun Servers and Open Source software. How to use the Olio web 2.0 toolkit to measure the performance and scalability of different deployment configurations so you can choose the best one before you deploy. New Product Pages

Jimmy Huang, Kemer Thomson, and Vicky Hardman added four new server pages to the Product area.

• Oracle Sun Blade Systems
• Oracle Sun SPARC Enterprise Servers
• Oracle Sun x86 Servers
• Oracle Sun Netra Carrier-Grade Systems

New From The Community

• CIS releases new security benchmark for recent releases of Oracle Solaris 10.
• Recover from a crash during a zone attach, by Philip Torchinski
• Adam Leventhal's farewell blog.
• Animation video: life's little instruction manual.
• Developer's Guide to Open Source Software Licensing
• PC Magazine's Coverage of Fowler's Strategy Session
• Server Watch: Oracle Details Upcoming Solaris 11 Release
• Rossi-Stoner Laguna Seca 2008 Battle Highlights
• Brendan Gregg's Pet Pterodactyl
• Documentation for Oracle Sun Storage Tek SL8500 Modular Library System
• Find out when a user last logged in and out

New Headlines

VirtualBox 3.2.8 Available for Solaris, Linux, Windows, MacOS

Get the latest maintenance release of VirtualBox software, version 3.2.8. VirtualBox lets your system run several OS's simultaneously. Great for testing new features before implementing. See this list of 3.2.8 enhancements.

Download the Oracle Solaris 10 10/09 Virtualbox Appliance Image

The Virtualbox Appliance image of Solaris 10 (10/09 release) is a great way to become familiar with the Solaris 10 OS inside the Virtualbox environment. The appliance image is free, and available in Open Virtual Format. Requires VirtualBox 3.0 and 5 GB free space.

Get 35% Off System Administration Training CD

For a limited time you can save 35% when you purchase the Sun System Administrator Library Self-Study CD course. Study for the certification exam on your own schedule. No travel required. To get the discount, contact your Oracle University Sales Rep and mention promotion code SSCD Promo 2. - Rick

I've been in touch with Robert C. Kelsey for a while. I met him on the dev mailing list, i.e., the mailing list for NetBeans Platform developers. He is the owner and programmer behind AMSWin: "AMSWin is software specifically designed for AMSOIL dealers. It allows business owners to manage their Customers, fellow Dealers, Inventory, Orders and Other Transactions in a way that ensures a manageable and efficient operation. It provides detailed reports that give the dealer a vivid picture of the status of the business."

The application is being ported to Java and, since it's a large application that needs to be flexible (not all users need all features, etc), to the NetBeans Platform. Robert sent me the current state of his application, which is created on top of JSR-296, the Swing Application Framework:

Less than a day later, it is now mostly ported to the NetBeans Platform:

The benefits? A much cleaner architecture, with all the sources neatly organized into modules:

Currently there's still a dependency on the Swing Application Framework, because one of the modules is still using the ResourceMap from that framework, which can easily be exchanged with the NetBeans NbBundle class.

It's pretty cool that Robert's original JDialogs continue to work as before. I.e., I simply copied them into a module, tweaked a bit to remove the ResourceMaps, and then I was able to call them from my Action classes.

Two special hidden features of the current state of the application are that (1) @ConvertAsJavaBean is used to serialize each newly created instance of the "BusinessAccount" object (with the upper ChildFactory listening to a folder for new entries and repopulating its node hierarchy displayed in the OutlineView) and (2) the Actions defined in the lower window (which, of course, is defined in a different module to where the upper window is found) come from the layer, so that anyone can easily extend the list of menu items from an external plugin: class OrderBasketBeanNode extends BeanNode { public OrderBasketBeanNode(OrderBasket orderBasket) throws IntrospectionException { super(orderBasket, Children.LEAF, Lookups.singleton(orderBasket)); setDisplayName(orderBasket.getOrdNumb()); } @Override public Action[] getActions(boolean context) { List<? extends Action> registeredOrderActions = Utilities.actionsForPath("OrderActions/"); Action[] actions = new Action[registeredOrderActions.size()]; for (int i = 0; i < registeredOrderActions.size(); i++) { Action oneRegisteredOrderAction = registeredOrderActions.get(i); actions[i] = oneRegisteredOrderAction; } return actions; } }

There are many other cool things about how all this is implemented. Maybe this example will be used as the porting example during the NetBeans Platform Certified Training in South Africa. It really illustrates many things about the NetBeans Platform.

The DPS 6.x and 7.x default configuration must be slightly changed to be able to work smoothly with OID. Indeed, the DPS health-check engine might compute OID operational state incorrectly and consider OID down when it is up and running.

There are 2 ways to fix that: either disable the proactive health-check monitoring or (recommended) change the LDAP query done to determine the backend operational state. To do so, make sure DPS is running and launch the command below on each ldap data source associated with an OID instance:

dpconf set-ldap-data-source-prop <data source> monitoring-search-filter:(objectclass=*)

Note: this configuration is not specific to OID and works well with other directory server instances, including DSEE.

Today I was preparing a customer project on my test server to run a different application (then the default Oracle VDI desktop) on some special Sun Ray DTUs. In a recent post I wrote about configuring Opera as a Sun Ray Webkiosk browser and this was the application we wanted to run on the special DTUs (for a reception area). In this article I explain how to configure this for the Webkiosk browser, but the scenario is the same for every application you have in mind for a kiosk session.

There are multiple ways to configure different kiosk applications in the Oracle VDI server environment:

1. Write your own Kiosk Mode scripts. The Think Thin Blog is a very good source to get your inspiration,
2. Use the Meta Kiosk add-on developed by Daniel Cifuentes,
3. Use the standard Kiosk Mode interface as described in Jörg's Desktop Blog.

I decided to use method three, it is a less known feature in the Sun Ray Server software (which is embedded in the Oracle VDI software), easy to configure via the CLI (unfortunately it is not yet integrated in the GUI) and you do not need to write a shell-script to control the multiple applications Kiosk Mode logic.

To refresh your mind from the earlier blog post about Opera: you need to install the Opera web-browser on your Oracle VDI servers and you need to store the kiosk description file and launcher in the directory /etc/opt/SUNWkio/sessions.

When this is done you let the Oracle VDI server know that for a pre-determined number of Sun Ray DTUs you do not want to use the default (Oracle VDI) kiosk session, but the alternative Opera Webkiosk session (or any session that you have in mind as alternative). This is a three step process:

1. First you have to store the Opera browser kiosk session configuration in the Sun Ray server data store. You have to create a little configuration file and use this file to store the information in the data store. After this is done you check if it is stored together with the default session

# I have created the WebkioskSession.conf with an editor. root@vdiserver:# cat WebkioskSession.conf KIOSK_SESSION=webkiosk KIOSK_SESSION_TIMEOUT_DETACHED=12000 root@vdiserver:# utkiosk -i WebkioskSession -f WebkioskSession.conf root@vdiserver:# utkiosk -l session WebkioskSession root@vdiserver:#

2. Then you register the token of the device (e.g. pseudo.00144f5787d1) or the token of a smart-card. Adding a name to the registration is mandatory, I use dummy01 as a not existing username.

root@vdiserver:# utuser -a "pseudo.00144f5787d1,,,dummy01," root@vdiserver:#

3. In the last step you override the default kiosk configuration by the alternative and restart the session by killing the current session for that token. 

root@vdiserver:# utkioskoverride -r pseudo.00144f5787d1 -s kiosk -c WebkioskSession root@vdiserver:# utsession -k -t pseudo.00144f5787d1 root@vdiserver:#

If you have multiple tokens (for example 20 Sun Ray DTUs in your reception area), then step two and three are very easy to script in a shell command or shell script. I let this to the user as this is out-of-scope for this article.

Everyone who uses daily builds of our PHP support in NetBeans IDE might noticed couple of new features that were added after 6.9.1. One of them is Type Hierarchy View which is helpful for OO PHP code. 

To show this view use action from editor popup menu: Navigate >  PHP Type Hierarchy.

One can assign shortcut for this action - use the same steps as described here.

Then you will see type hierarchy in window next navigator like this:

 Two buttons let you choose one of two views:

• Subtype hierarchy - displays class and its subclasses or interface and the classes that implement it (on the picture above)
• Supertype hierarchy - displays classes that are extended and interfaces that are implemented 

 All the issues or enhancements please report in NetBeans Bugzilla

Tom Haynes

Originally posted on Kool Aid Served Daily
Copyright (C) 2010, Kool Aid Served Daily

On a recent trip (my first as an employee!) to Oracle HQ, I decided to test out Oracle VDI 3.2 from an airplane.  Sure, everyone (who can...if the latency will allow their protocol to work that is) has done this, but hey it's been my first trip since being part of the Big O.  One could say I don't travel like I used to. 

Unfortunately, I was a bit ill prepared to do this demo from a battery standpoint on my video camera, but still managed to get some screen caps with my phone.  On my next trip I'll bring my flip fully charged to get the full sequence.  But I'll tell you, from some 30K thousand feet, 300 ms (plus) latency, it did awesome.  While it won't surprise most to know I really believe in our technology, I did all of this on my own dime.  From the the gogo inflight internet to the first class upgrade.  Hey, us tall folk need space to work!

 My Building:  (The "Big Building" if you know what I mean)

Somewhere over the Mojave:

And Let's Connect:

Picking my Windows 7 Pool: 

And here we are: 

This is the Virtual Desktop Client in windowed mode. It's hard to make out the latency, but it's over 300: 

The new "IBM Power Systems Performance Report POWER7, POWER6 and POWER5 results" holds an interesting piece of information. A reoccuring question be colleagues and befriended admins is the impact of LPARs to the performance. It looks like IBM needs the LPARS to get some speed out of their larger systems.

Just a few examples: When using a 795 with 4.25 GHz and 64 cores a configuration with 4 LPARS a 16 cores yield a relative performance of 926.28. The same system with just 1 LPAR with 64 processors yield a relative performance of 777.09. So leaving the scaling to the OS instead of dividing it into 4 small systems gives you just 83.89% of the performance. When using a 795 with 4.25 GHz and 64 cores a configuration with 8 LPARs with 16 cores each yields a rperf value of 1852,56. With 2 LPAR with 64 cores each you get 1554,18. Interestingly is 83,89% again.

At first i thought "16 cores are easily fitting on a processor book (with 4 procs each). A 64 cores LPAR has to use two processor books. So when you use a configuration larger than a processor book you will leave 16,11% on the way". But doing the same calculation with some other data showed otherwise. But the move from 32 to 64 core lpars just reduce the performance by 5,7 percent respectively 5,4 %. 32 cores fit on a processor book, too. Thus the difference should be similar to the 64 to 16 cores situation.
So my interpretation is a little bit different: The scalability of AIX seems to have sweet spot between 16 and 32 cores. I thought a moment about an intra-book bottleneck, but the CPUs on the book are fully meshed (1 hop from each CPU to every other), so i don't think it's a problem.



When you look into this chart (please click into the image for a larger version), you may find some interesting points. The light blue line is a hypothetical perfectly scaling 4.0 GHz Power7 in a 795. The data is based on rPerf number of 103.41 for an 8 core system(source: Page 20 of the document). Please look at the right side of the chart at 256 cores. You end up at 81,9% of the hypothetical performance when you use 64 core LPARs and at 86% of the hypothetical performance when using 32 core LPARS (the difference is interestingly pretty much the same as computed before for 64 cores instead). At 64 cores your load is distributed at 8 processors, thus just 2 processor board. Still there is an serious impact of almost 20%. Will be interesting to further dig into this topic.

However it's important to know, that the operating system is limited to 64 core SMP no matter how many cores are in the system by the LPARs configuration. So this numbers doesn't factor in scalability challenges of AIX above 64 cores as the os has not to scale above this point while generating this rperf numbers. The numbers for the large core number configurations are not single OS image numbers. Further penalties for the OS scaling comes on top. Furthermore this benchmark is a pure CPU/memory benchmark. As IBM explained in their own description of the benchmark, there is no I/O and no networking involved.

That said, a number of really interesting data points are missing in the pdf:

• The rperf number of a fully blown unpartitioned system
• The rperf number on a fully blown system with just one LPAR in the size of the complete system
• Somehow i have the impression, IBM is hiding something. When you look into the mentioned pdf there is SPEC number for an unpartitioned system, but no rperf for it. The existence of the SPEC numbers hints to the point, that they had indeed an OS that was able to scale to 256 cores. On the other side, there is no SPEC number for the partitioned systems, but the rperf numbers. Just call it a presentiment ...

Yet another one leaves Oracle: Adam Leventhal.

A few years ago I wrote about the "Importance of MTU" as it pertains to Sun Ray clients.  Things changed with the Sun Ray and in most circumstances, we can detect and adjust the Path MTU accordingly.  While the Oracle Virtual Desktop Client (OVDC) is designed to be just like a physical Sun Ray, for various reasons it currently lacks the Path MTU Discovery (PMTUD) code.  In short, what's old is new again..Or is it what's new is old again.  Either way, you need to set the MTU for the best experience (improper MTU demo video below).

You can set the MTU with a slider from the network tab:

Unfortunately the slider isn't as precise as you might need it and seems to jump in blocks ranging from 6-8 (i.e. try to set yours like mine with an MTU of 1333 and good luck!).  If somewhere within that range lies the your value, you can use the command line to set a precise MTU.  Simply run the vdc command with the --mtu argument.

For Windows:

C:\Program Files\Oracle\Virtual Desktop Client\vdc --mtu 1333

For OS X:

/Applications/Oracle Virtual Desktop client.app/Contents/MacOS/vdc --mtu 1333

The MTU setting is sticky and will stay for all subsequent connections, unless of course you use the slider and change it.

Quick (no audio) Video of what an improperly set MTU looks like:

Your browser cannot play this video. Learn how to fix this.

Yesterday and today I worked through a lot of the FAQs, mailing lists, and other random documentation on the web in order to figure out a simple procedure for setting up unit tests, functional tests, and code coverage measurement for a NetBeans Platform application. Turns out, it was really easy, since all the related frameworks are part and parcel of the NetBeans Platform build harness. Simply go to the "harness" folder in your NetBeans IDE installation and you'll see what I'm talking about. Everything from extensions to Jemmy and JUnit for NetBeans Platform applications, to code coverage via Cobertura.

So I wrote this new tutorial, describing it all:

http://platform.netbeans.org/tutorials/nbm-test.html

I sent it around to a few people to review it and I got this interesting e-mail back from NetBeans Platform guru Tom Wheeler:

"One thing that's noteworthy here (perhaps blogworthy) is that complete support for testing a platform application, including unit testing and functional testing -- plus reports showing you how effective that testing is -- comes "for free" when you build on the NetBeans Platform.

With a regular Swing application, you usually have to go set up your Ant build file to compile, package, run, test and measure test coverage for the application. This takes a long time to do when starting from scratch and is even still tedious to try and reuse build files from some previous project.

This has already been done for you when you build on the platform, which means that the NetBeans Platform is a good choice for even small applications, because you'll spend less time writing build files and more time writing code. The platform's modular architecture will make it easier to maintain your application as it grows."

I have a few tweaks to make to the tutorial, a few comments to process, but, really, it's all a pretty amazing thing. Forget about setting up unit test infrastructures, functional test infrastructures, and code coverage infrastructures, because the NetBeans Platform provides them out of the box. You simply need to use the existing infrastructure, rather than setting it up yourself. That's the point of the above tutorial.

I joined the Solaris Kernel Group in 2001 at what turned out to be a remarkable place and time for the industry. More by luck and intuition than by premonition, I found myself surrounded by superlative engineers working on revolutionary technologies that were the products of their own experience and imagination rather than managerial fiat. I feel very lucky to have worked with Bryan and Mike on DTrace; it was amazing that just down the hall our colleagues reinvented the operating system with Zones, ZFS, FMA, SMF and other innovations.

With Solaris 10 behind us, lauded by customers and pundits, I was looking for that next remarkable place and time, and found it with Fishworks. The core dozen or so are some of the finest engineers I could hope to work with, but there were so many who contributed to the success of the 7000 series. From the executives who enabled our fledgling skunkworks in its nascent days, to our Solaris colleagues building fundamental technologies like ZFS, COMSTAR, SMF, networking, I/O, and IPS, and the OpenStorage team who toiled to bring a product to market, educating us without deflating our optimism in the process.

I would not trade the last 9 years for anything. There are many engineers who never experience a single such confluence of talent, organizational will, and success; I'm grateful to my colleagues and to Sun for those two opportunities. Now I'm off to look for my next remarkable place and time beyond the walls of Oracle. My last day will be August 20th, 2010.

Thank you to the many readers of this blog. After six years and 130 posts I'd never think of giving it up. You'll be able to find my new blog at dtrace.org/blogs/ahl (comments to this post are open there); I can't wait to begin chronicling my next endeavors. You can reach me by email here: my initials at alumni dot brown dot edu. I look forward to your continued to comments and emails. Thanks again!

The GlassFish Giftshop has been around since 2007; it currently includes several t-shirts, mugs, stickers and Sigg Water Bottles.  JavaOne (our "best"-selling season?) is around the corner, so we want to refresh the graphics and perhaps adjust the items.

Give us feedback if you want additional items in the store - we can easily add any from the inventory of CafePress - or if you think we should drop any, or if you want us to consider any special graphics to place on the items.

Just a few copy&paste commands, and a bit of CSS fine tuning, were necessary to relaunch Jaap’s blog on VDI. Who’s next?

• Jaap’s VDI Blog Space...
• Think Thin...
• Uw Werkplek, maar dan anders...
  

Congratulations to IBM for the new lead in TPC-C. IBM has once again the lead in a benchmark that got meaningless a few years ago. Just before you ask, i wrote the same when Oracle was first. But somehow it looks like IBM wasn't able just to call it a day So they did a TPC-C benchmark run again and at the end they were able to yield 10,366,254 tpmC.

The result is indeed impressive. However there is are some key difference. The response times are vastly longer in the IBM result.

Response Times
(in seconds)90th %AverageMaximum New Order2.11.13724.041 Payment2.11.13821.293 Order-Status2.061.09520.169 Delivery (interactive)1.640.74917.953 Delivery (deferred)0.950.422.48 Stock-Level2.081.11321.547 Menu1.640.7723.037
Now look at the response times at the Oracle result.
Response Time
in Seconds90 th %Avg.Max. New-Order0.1700.1685.885 Payment0.1600.1565.758 Order-Status0.1500.1505.433 Delivery (Interactive)0.1200.1343.869 Delivery (Deferred) 0.0400.0212.839 Stock0.2100.1824.796 Menu0.1200.1364.474

With similar response times, the number of transaction is more in the range of 8,9 million as stated by the diagrams in the full disclosures.

This is the diagram from the IBM result:

Somewhere between 80% and 100% of the final result the reaction time explodes.

This is the matching result of the TPC-C result of Oracle:

There is no equivalent "explosion" in this result....


Furthermore i want to hint you on certain points in the configuration as stated by the full disclosure.

• This configuration has 48*380 GB SAS cards with battery backed write cache, thus 17,8 GiB battery protected write cache in total.
• The configuration used 3*224 SSDs summing up to 672 SSD. I would suspect that they use SSD with Sandforce 1500 (the same 177 GB like they used in the TPC-C that is documented at several places to be generated with Sandforce based SSD). This controller has an interesting capability. It's capable to do compress and some benchmarks have suggested, that the performance of this drive is quite different with compressible and less compressible data. It's would be an interesting point to research how compressible TPC-C data is.
• The SSD use MLC. The Sandforce Controller have some special mode of operation to enable the use of MLC to reach better durability, but this mode is based on compression, too. Interesting questions are: Are the SSD really capable to hold 3 years of TPC-C load due to the usage of MLC and what would be the impact on durability of less compressible data.
• The database is completely on the SSD. Just the database log is on disk. That's similar to the Oracle config
• The configuration of the database on the system is ... well ... interesting. They use a partitioned database and all this partition are bound to certain resource sets. So essentially they splitted the system in several ones, to be exact ... into 32 partitions per systems bound to a resource set each. With this amount it's ensured that all requests are CPU local factoring out the interconnect.
• The configuration doesn't provide any availability protection, but that's okay, because TPC-C doesn't mandate such. Just keep this in mind with the pricing and with the configuration. There are no mechanisms ensure availability and you can't transform it into an available configuration because the storage is direct attached to a single node (the storage is in the i/o drawers). The Oracle configuration is highly available by accident, as it uses shared storage and RAC

Conclusion
I think TPC-C is now really a corporate ego thing. We are in an arms race here. I'm really interested, how Oracle strikes back