Managing REDO log performance

I have written before about managing database performance issues, and the topic is hot and alive as ever. Even with today’s fast processors, huge memory sizes and enormous bandwidth to storage and networks.

warning: Rated TG (Technical Guidance required) for sales guys and managers ;-)

A few recent conversations with customers showed other examples of miscommunication between IT teams, resulting in problems not being solved efficiently and quickly.
In this case, the problem was around Oracle REDO log sync times and some customers had a whole bunch of questions to me on what EMC’s best practices are, how they enhance or replace Oracle’s best practices, and in general how they should configure REDO logs in the first place to get best performance. The whole challenge is complicated by the fact that more and more organizations are using EMC’s FAST-VP for automated tiering and performance balancing of their applications and some of the questions were around how FAST-VP improves (or messes up) REDO log performance.

Read more of this post

Performance – The database stack

hamb-stackAs mentioned before, I frequently find myself in discussions around Oracle performance and how an Oracle database behaves on EMC storage. It turns out that often there is a lot of confusion on how the different layers interact with each other and very few people seem to understand the whole stack.

So I started a personal challenge to make a “one picture tells more than 1000 words” complete overview of the Oracle on EMC database stack.

I failed.

Turns out it’s nearly impossible to get everything in one picture without cutting corners.

So here is a simplified (and therefore incorrect) picture. It ignores certain complexities and is far from complete, and might even contain errors.

Read more of this post

Application processing at lightning performance – The hourglass view of access times

HourglassEven in these modern times, when lots of things are changing in the ICT world, some lessons from the past still hold true.

Previously, I discussed the I/O stack in a typical database environment. As virtualization has complicated things a bit, the fundamental principles of performance tuning stay the same.

Recently I was browsing through old presentations of colleagues and found another interesting view on response times in an application stack. Again, I polished it up a bit and modified it to reflect a few innovations and personal insights.

The idea is as follows. We as humans have problems getting a feel of how fast modern microprocessors work. We talk in milliseconds, microseconds, nanoseconds. So – in the comparison we assume a 1 Gigahertz processor and then scale up one nanosecond to match one second – because this fits better in human’s view of the world. Then we compare various sorts of storage on the “indexed” timescale and see how they relate to each other.

Read more of this post

Performance – The I/O stack

Concorde Mach Indicator

In my last post, I gave a highly simplified representation of “The I/O stack” of a database. In reality, it’s much more complex.

I found an old picture where the whole I/O stack of a database was described and I decided to brush it off and include some additional layers (application and middleware) and show how the I/O flows if you run on a virtual server with a hypervisor.

Also, the storage network can provide virtualization which in turn adds a layer of complexity.

Read more of this post

The missing link in application performance tuning

When dealing with application performance problems, the quick and easy solution is often to throw hardware at the problem. Typically this is one out of processing power (CPU), memory, or more I/O resources. With a bit of luck, the bottleneck is removed and shifts somewhere else (but now the total stack is – hopefully – capable of handling more workload).

Missing Link

Read more of this post

Managing database performance SLA’s with quality of service

A guy walks into the showroom of a Porsche dealer. He wants to buy a new set of hot wheels. The sales guy tells him about the latest technology in sports car design. This year’s model has active 4-wheel drive traction control, a very powerful engine (over 500 horsepower) with direct fuel injection, semi-automatic dual-clutch with seven speeds, and the whole car is weight-balanced to offer the best handling and cornering speeds. At the same time, carbon emissions per kilometer are the lowest in years and the car actually has green labels, so at least you can make yourself believe that it does not ruin the environment too much ;-)


“Great,” says the customer. “What’s the acceleration and top speed?”

Read more of this post

Pinpointing I/O bottlenecks on Linux

Does this story sound familiar?

The end users of a database application start complaining about poor system response and long running batch jobs. The DBA team starts investigating the problem. DBA’s look at their database tools such as Enterprise Manager, Automatic Workload Repository (AWR) reports, etc. They find that storage I/O response times are too high (such as an average of 50 milliseconds or more) and involve the storage team to resolve it.

The storage guys, in turn, look at their tooling – in case of EMC this could be Navisphere Analyzer, Symmetrix Performance Analyzer (SPA) or similar tools. They find completely normal response times – less than 10 milliseconds average.

The users still complain but the storage and database administrators point to each other to resolve the problem. There is no real progress in solving the problem though.

Two Way Communications

Read more of this post