ORACLE-BASE Blog Aggregator » Doug Burns

Randolf Geist on 11g Incremental Statistics

Doug Burns — Tue, 17 Jan 2012 07:12:00 +0000

Well it wasn't the post I planned to return to technical matters with.

Lots of readers here have asked me when I'm going to get round to writing about 11g Incremental Statistics as part of the stats series. Although Incrementals are on my To Do list, I wanted to finish off the stats copying posts first. In any case, Randolf Geist got there already so I'll cross it off my list and point you towards his post instead.

Yes, I know there have been a lot of Incrementals posts already by people like Robin Moffat and John Hallas, but Randolf's post maps most closely on to the post I planned, which is an overview of Incrementals that highlights some of the practicalities of using them in "The Real World". I'd particularly draw attention to a couple of aspects which I think people keep misunderstanding.

The first time you implement Incrementals on a table, Oracle will have to trawl through the entire table in order to build the initial synposes. This has always seemed obvious to me - how can you incrementally build on synposes that haven't been created yet? But the long duration initial gather seems to surprise people and they decide that Incrementals are 'slow'.
Incrementals are a replacement for GRANULARITY=>'GLOBAL AND PARTITION' and not 'PARTITION'! Expecting an option which gathers Partition stats and then goes around updating synposes to perform as well as a simple partition gather is unrealistic^*. Any performance improvement needs to be measured against both gathering the Partition stats and maintaining the Global stats. Incrementals will almost definitely be quicker than that! I prefer to think of Incrementals not so much as a performance improvement (because most people probably didn't regather Global statistics every time they gathered individual Partition statistics because they didn't have the time on an active system), but an improvement to the quality of your Global stats because you can now afford to maintain them with the same frequency as your Partition stats, rather than scheduling an out-of-hours Global stats gather or depending on the inaccurate NDVs that result from the previous aggregation process.

Good post, anyway. Thanks Randolf!
^* However, it's fair to say that Oracle have continued trying to improve the performance of synposis maintenance.

Upcoming Conferences

Doug Burns — Mon, 16 Jan 2012 05:45:00 +0000

At this time of year, I'm usually panicking about my Hotsos presentation/paper and up to my eyeballs in test scripts. I enjoy it and the absence feels a little strange, but I decided to do something different this year.

Although it's the 10th Symposium and I'm sure it will be as educational and fun as ever, I decided not to submit an abstract for a variety of reasons, one of which was the cluster of conferences that occur during the spring. Although the Oracle ACE Director program helps with travel expenses, the rest is self-funded including the time off work so I need to pick and choose my conferences carefully. For years Hotsos, Openworld and the UKOUG were fixtures but that means I've missed out on the wonders of Miracle spring events and the various Scandinavian boat trips!

Of course, almost as soon as I decided I wouldn't be going to Dallas the regret started to set in as there are people who I often only see at Hotsos who I won't be able to catch up with thise year and I think some of my best presentations have been there. It got worse when I looked at the list of confirmed speakers and spotted an old friends name. Sigh, having encouraged Paul Matuszyk to attend the Symposium while we were working together in Scotland he enjoyed attending a few times and decided he wanted to present there one day. Now that he has been accepted, I won't be there to see it (By the way, this reinforces a point I've made to several people. Although you will see a lot of familiar faces presenting, there's always been space for new presenters as far as I'm aware. Mark Rittman is another 'new' speaker there this year.)

However, now that the decision's been made, I need to let it go and move on to those conferences I hope to attend instead.

I had a terrific time at the OUGN (Norwegian User Group) spring conference on the boat last year so I was delighted when they invited me back to speak at this year's event.

My presentations there will be an existing one from OOW and UKOUG and a new one ...

Performance and Stability with Oracle 11g SQL Plan Management

This presentation discusses the use of Oracle 11g SQL Plan Management to safely upgrade a complex and critical Data Warehouse system, ensuring stable performance whilst taking full advantage of optimizer improvements in Oracle Database 11gR2.

Subjects covered include -

SQL Plan Management fundamentals
Controlling execution plan evolution
Using SQL Plan Management to resolve complex performance challenges
The interaction between bind variables, histograms, Adaptive Cursor Sharing and SQL Plan Management.

The presentation will focus on real world experiences of both the strengths and limitations of SPM.

Falling in love all over again - OEM 12c Performance Page Enhancements
Oracle RDBMS 10g introduced significant instrumentation enhancements and the OEM 10g pages illustrated them to make performance analysis much easier. OEM 12c is the latest evolutionary stage of the Performance Pages and this
presentation shows the new features that make them even more compelling and useful in more situations.

I'm also hoping to attend Miracle Openworld because it's a long time since I was in Denmark and I realised last year just how much I've missed this conference. I submitted the Statistics on Partitioned Objects abstract because I think it would be good for that particular conference and because it's still what I'm spending most of my time on in the office at the moment. Having said that, I haven't heard anything back yet and it looks like the agenda already has some pretty interesting presentations on there, so I might not make the cut this time.

Still tempted to go, though

UKOUG 2011 – Ash Outliers

Doug Burns — Tue, 10 Jan 2012 00:19:00 +0000

My final UKOUG 2011 post is about another of my favourite presentations - "ASH Outliers: Detecting Unusual Events in Active Session History" by John Beresniewicz. (JB for short, but Marco Gralike made a fairly good stab at pronouncing his surname correctly during the introduction.)

I'd been looking forward to this presentation because I'd already been aware of his work in this area for a while and was supposed to be helping out (but I'll come back to that later ...). I'd also expected to see it in 2010 but JB withdrew the abstract later. Although I knew a lot of the content already I enjoyed it because of the subject and because JB's always likely to make me see something old from a new angle. Anyway, what was it all about?

One of the most important limitations of ASH data is that, because the data is sampled, it doesn't include every individual timed event that sessions waited for and is inherently biased towards longer duration (and more common) events. As long as you understand this design decision then you can understand sensible ways to use the data and also less sensible ways!

For example, it is not sensible to write queries against ASH data that SUM(time_waited) or AVG(time_waited) because the results look sensible but are fairly meaningless. If you sum all the wait times in ASH but ASH doesn't include all activity then what does the total value represent? Likewise, if you calculate the average 'log file parallel write' time in ASH for a specific period, it's virtually guaranteed that the result will be higher than the true average because ASH will tend to capture more long duration events than short ones.

Generally it's more sensible to use simple COUNT functions and make the assumption that a sample represents a second of time. It's obviously an approximation, but one that works surprisingly well as reasonably long experience of the OEM performance pages and ASH queries has shown me.

However, although the Oracle employees I've discussed this with encourage people to use the TIME_WAITED column in ASH with caution, I've personally found it useful on many occasions when ASH is the only session-level data available because I'm trying to diagnose a problem after the event. In fact, ASH can be extremely useful in that situation because, as well as having session-specific information, it contains it for all the sessions. So I confess that when querying ASH data using SQL, I've often found the TIME_WAITED of individual rows to be a useful indication of what's been going on if unusual or particularly long waits appear immediately prior to a serious problem. By looking at the series of events that led up to a problem, I can see the interactions between those sessions via blocking session information and at least build up an approximate view of what happened.

JB's presentation was essentially about a single query that he's been working on that is designed to look at the TIME_WAITED values in ASH data to identify outliers - events with unusually long wait times relative to the usual wait times for that event. The thinking being that the appearance of long wait times for certain events could indicate the root cause of serious system-wide performance problems. (Note that AWR data would be next to useless for this because the problems might only occur for a few seconds in the lead-up to the problem and that level of detail would be aggregated to insignificance in the wider AWR time scope.)

Having hopefully said enough technical things in this post now to keep grumpy people happy, allow me a humorous interlude. From the start of the presentation, JB kept alluding to a couple of things

a) The singular lack of response he'd had from people he'd shared the query with. Fortunately no names were mentioned but he was clearly griping about the limited response he'd had from me after sending me the query a while back and how long it had taken. What can I say? I'm *busy*!

b) Why the feedback is so important to him - because people in Oracle Development appear very interested in what customers are doing on their real Production systems. Even though Oracle do have their own internal Production systems, they're only a small fraction of what customers are doing so it's difficult to know what will work. In any case, I guess Oracle don't let developers play around on their production systems, like most companies.

So here's the deal. As JB mentioned, he is probably the least on-line guy there is. He might do DBA 2.0, but he doesn't do Web 2.0! So he suggested that I could get the script out there to other people to try on their own interesting systems and see whether they found it useful.

Here it is

Note that this version references the V$ rather than GV$ version of ASH data so only uses officially supported features as JB alluded to during the presentation. i.e. a RAC version is possible but this is single-instance for now, just to let people give it a try.

To save a veritable deluge of mail (yeah, JB, you *wish*! LOL) it might be best to post any feedback on the script as comments here or I could forward it on.

Oh, and something I heard later that a few people had struggled with during the presentation was the concept of Significance Levels that are central to the script and the presentation. Fortunately, I already host a short document on the subject, again courtesy of JB, here. However, as that's more focussed on Adaptive Thresholds and time grouping in particular, you might find the discussion on this AskTom thread more generic and relevant to the ASH Outliers script.

All in all, a terrific presentation, as I expected and something that people might want to try out. It’s certainly an interesting concept and an attempt to automate something I’ve been doing manually for a few years. Thanks, JB!

UKOUG 2011 – Cary Millsap’s Keynote Videos

Doug Burns — Mon, 09 Jan 2012 00:05:00 +0000

I noticed last week that the keynote presentation I discussed in a previous post is available on YouTube, in 5 parts ....

Part 0
Part 1
Part 2
Part 3
Part 4

It was a strange experience watching it again because, although I recognise it as precisely the same presentation, it didn't have as much impact on YouTube. Perhaps Cary has some sort of strange Jobs-like Reality Distortion Field when you're in the same room?

Extended DISPLAY_CURSOR With Rowsource Statistics

Doug Burns — Sun, 08 Jan 2012 23:45:15 +0000

Just a quick post to draw attention to Randolf Geist's post here. Good stuff.