Video : DBMS_CLOUD : Objects and Files

In today’s video we’ll demonstrate the functionality of the DBMS_CLOUD package, with specific reference to objects in a cloud object store and files on the database server file system.

The video is based on part of this article.

You might find these useful also.

The star of today’s video is Roel Hartman, who used to do APEX, but now just runs marathons I think… πŸ™‚

Cheers

Tim…

Data Pump Enhancements in Oracle 21c (and a little support story)

I’ve been having a play with some of the Oracle 21c data pump enhancements. This post links to the resulting articles, and includes a little story about one of the features.

The Articles

Here are the articles I wrote during this investigation.

As an aside, I also knocked up a quick overview of the DBMS_CLOUD package. I’ve used many of the routines in this package in my autonomous database articles over the last few years, but I got a bit sick of jumping around to get the syntax of different operations, so it seemed sensible to have a centralised description of everything, along with working examples.

The Story

The article about using expdp and impdp with a cloud object store (Oracle Cloud Object Storage, AWS S3 or Azure Block Storage) came with a little bit of drama.

Back in the 18c days it was possible to import into an Autonomous Database using a dump file on a cloud object store using the 18c impdp utility. I wrote about this at the time (here). At that time export to a cloud object store using the expdp utility was not supported, and the import wasn’t supported with an on-prem database.

Oracle 21c introduced the ability to export from an Autonomous Database to a cloud object store, which worked fine first time. The documentation also listed a new feature called, “Oracle Data Pump Supports Export to and Import From Cloud Object Stores“. This sounded very much like it meant for on-prem databases, and sure enough it did.

When I started trying to use this feature I pretty quickly hit a road block. The expdp utility couldn’t connect to the object store bucket. I raised a call with Oracle Support about it. While I was waiting for a response I figured this functionality may have a dependency on the DBMS_CLOUD package under the hood, so I installed it in my on-prem database. The on-prem installation of DBMS_CLOUD was working OK, but the expdp utility was still failing to contact the object store bucket.

Due in part to friends in high places, my SR got picked up and it was confirmed the DBMS_CLOUD installation was an undocumented prerequisite, but it was still not working for me. The support engineer confirmed they could replicate the issue too. A few interactions between support and development resulted in bug 33323028, which fortunately had a simple workaround. At that point the support engineer was up and running, but I still had a problem. A bit of tracing later and it turned out my remaining issue was PEBCAK (Problem Exists Between Chair And Keyboard)…

When I installed the DBMS_CLOUD package it said to put a wallet reference in the sqlnet.ora file. I did that and the package seemed to be working OK, so I thought everything was good. Unfortunately I put it under the ORACLE_HOME and Oracle 21c uses a read-only Oracle home, so that’s the wrong place. It didn’t affect the package, as that picks up the wallet location from a database property, but it affected the expdp and impdp utilities. I keep telling people read-only Oracle homes will trip you up if you are not used to them, and sure enough it tripped me up. Damn you muscle memory! Once the correct sqlnet.ora file was amended everything was good.

So the journey to get this feature working involved:

  • An undocumented prerequisite, which I guessed.
  • A bug which Oracle Support and the dev folks gave me a workaround to.
  • An idiot (me) trying to learn not to be an idiot.

With a bit of luck the bug fix/workaround will be rolled into a future release update, so you may never see this. The MOS note about the DBMS_CLOUD package installation suggests this might also be part of the database by default in future. That would be great if it happens.

Anyway, after that little drama I was able to export data from my on-prem database to a dump file located on a cloud object store, and import data from a cloud object store into my on-prem database. Happy days!

Thanks to the support and dev folks who helped get me through this! πŸ™‚

By the way, all the other Oracle 21c data pump new features worked without any issues.

So there you have it. Some new articles and a little bit of drama… πŸ™‚

Cheers

Tim…

Video : Multivalue Function-Based Indexes for JSON_EXISTS in Oracle Database 21c

In today’s video we demonstrate multivalue function-based indexes for JSON_EXISTS, introduced in Oracle 21c.

The video is based on this article.

You may also find these helpful.

The star of today’s video is my sister-in-law and the queen of the database Maria Colgan. It’s a pleasure to be one of your unworthy minions… πŸ™‚

Cheers

Tim…

Video : DML Error Logging

In today’s video we demonstrate DML Error Logging, introduced in Oracle 10.2.

The video is based on a section of this article.

The star of today’s video is Oren Nakdimon, who’s chilling in his garden along with the birds. πŸ™‚

Connor McDonald has done some videos on this feature, which are worth checking out.

Cheers

Tim…

Database Patching Revisited : Take off and nuke the entire site from orbit…

I was reading a post by Pete Finnigan the other day.

I put out a tweet mentioning it, and linking to one of my old posts on the subject too.

This started a bit of a debate on Twitter about how people patch their databases. In this post I want to touch on a few points that came out Pete’s post an some of the other Twitter comments.

You have to have a plan!

An extremely important point made by Pete was you have to have a plan. That doesn’t have to be the same for everyone, and there may be compromises due to constraints in your company, but that doesn’t stop you making a plan. Your plan might be:

  • We will start a new round of patching immediately when a new on-off patch is released, and every quarter with the security announcements. I can’t see how this is possible.
  • We will patch every quarter with the security announcements. That’s what my company does.
  • We will patch once per (six months, year etc.)

Hopefully your plan will not be:

  • We will never patch and person X will take the blame when we have a problem.

Release Updates (RUs) or Release Update Revisions (RURs)

Database quarterly patches are classified as release updates (RUs) and release update revisions (RURs). First let’s explain what they are.

  • Release Updates (RUs) : These are like the old proactive bundle patches. They contain bug fixes, security fixes and limited new features. Let’s call that “extra stuff”. In 19c the blockchain tables and immutable tables features were introduced in RUs. Backporting and new features can introduce new risks.
  • Release Update Revisions (RURs) : These are just bug fixes and security fixes. In theory these are safer than RUs as less new stuff is introduced, but… See below.

So from first glance you are saying to yourself I want the safest option, so I want to go for RURs. The problem is RURs aren’t like the old security patches that you could continue applying forever. Ultimately you have to include all the “extra stuff” from the previous RUs, but you get the option of doing it later. This page in the documentation explains things quite well.

This table from that link is quite useful, showing you what version you will be on during a quarterly patching cycle.

What does this mean?

  • If you patch using the RUs, you are going to the latest and greatest each quarter.
  • If you use RUR-1, you are constantly 1 quarter behind on the RUs extra content, but you add in the missing bug fixes and security fixes using the RUR-1 patch.
  • If you use RUR-2, you are constantly 2 quarters behind on the RUs extra content, but you add in the missing bug fixes and security fixes using the RUR-2 patch.

In all cases you have the latest bug fixes and security fixes. You are just delaying getting the “extra bits”. So at first glance it seems like you might as well go with the RUs. The issue is some of the RUs are a bit buggy. If you go for the RUR-1 or RUR-2 there is a chance the bugs introduced in the base RU have been fixed in the subsequent RURs for that RU. So we could say this.

  • RUs: Oracle have zero time to identify and fix the bugs they’ve introduced in the RU.
  • RUR-1: Oracle have 3 months to find and fix the bugs they’ve introduced in the base RU.
  • RUR-2: Oracle have 6 months to find and fix the bugs they’ve introduced in the base RU.

I tend to stick with the RUs, although I am considering changing. Ilmar Kerm said he’s found RUs too buggy and tends to stick with the RUR-1 approach. I guess a more conservative approach would be to stick with the RUR-2 approach.

Your experience of the RUs verses the RURs will depend on what features you use, what extra stuff Oracle decide to include in the RU and what they break by including that extra stuff. The biggest problem I got was 19.10 breaking hot-cloning of PDBs, which was kind-of important. If I had used the RUR-1 approach I would never have seen that issue. Different people using different features see different bugs.

How good is your testing?

The biggest factor in the decision of which approach to take is probably the quality of your testing.

  • If your testing of applications against new patches is good, you can probably stick with the RUs. If the RU fails testing, go with the RUR-1 that quarter.
  • If you just work on the “generally considered safe” approach, meaning you apply the patches and don’t do any testing, maybe you should be using the RUR-1 or RUR-2 approach!
  • The ultra-conservative approach would be to stick with the RUR-2 approach.

Just patch!

Regardless of which approach you take, you’ve got to have a plan, and you should be patching. I know some of you don’t care about patching, and you are fools. I know some of you would like to patch, but your companies are dinosaurs. All I can say to you is keep trying.

In my current company we never used to patch. I spent years sending out quarterly reports summarising all the vulnerabilities in our systems and still nothing. Eventually a few other people jumped on the bandwagon, we had a couple of embarrassing issues, and the constant threat of GDPR gave us some more leverage. Now we have a quarterly patching schedule for all our databases and middle tier servers. We are not perfect, but it can be done.

Even now, we still have questions like, “can we miss out this quarter?”, but we push back very hard against this. One quarter becomes two, becomes three, becomes never.

New patches on the 20th July (see here). Good luck everyone!

Cheers

Tim…

PS. If you are not patching externally facing WebLogic servers you might as well close your company now. You have already given all your data away. Good luck with that GDPR fine…

Database Upgrades : It’s been a long road getting from there to here

Please play the Star Trek – Enterprise theme while you are reading this post. πŸ™‚

I’ve mentioned database upgrades a few times over the last year or more. Like many others, we are pushing hard to get everything upgraded to 19c. Over the last couple of weeks a bunch more systems got upgraded, and we are now looking like this.

The remaining 11.2 and 12.1 databases are all in various stages of migration/upgrade. I would not curse us by giving a deadline for the final databases, but hopefully soon!

The reason for mentioning that theme song is it starts with the words, “It’s been a long road getting from there to here”, and that is exactly how it feels.

Many of the database upgrades are technically simple, but the projects surrounding them are soul destroying. Getting all the relevant people to agree and provide the necessary resources can be really painful. This is especially true for “mature” projects, where the, “if it ain’t broke, don’t fix it”, mentality is strong. I wrote about the problems with that mentality here.

I’m not going to give you any blinding insights into how to do your database upgrades, because every upgrade is potentially unique, as I discussed here.

We always go for the multitenant architecture (CDB/PDB) unless there is a compelling reason not to. I think we only have one non-CDB installation of 19c because of a vendor issue. None of our other 3rd party applications have had a problem with using PDBs, provided we’ve made sure they connect with the service, not a SID. We don’t use the USE_SID_AS_SERVICE_listener_name parameter. I would rather find and fix the connection issues than rely on this sticking plaster fix.

In know I’ve said some of these things before, but they are worth repeating.

  • Oracle 19c is the current long term release, so it’s going to have support for a longer time than an innovation release.
  • Oracle 21c is an innovation release. Even when the on-prem version does drop, you probably shouldn’t use it for your main systems unless you are happy with the short support lifespan.
  • I recently heard there won’t be an Oracle 22c, so the next release after Oracle 21c will be Oracle 23c, which is currently slated to be the next long term release.

In short, get all your databases to Oracle 19c, and you should probably stick there until Oracle 23c is released, unless you have a compelling case for going to Oracle 21c.

Cheers

Tim…

Video : DDL_LOCK_TIMEOUT : DDL With the WAIT Option

In today’s video we demonstrate the DDL_LOCK_TIMEOUT feature, introduced in 11g.

The video is based on this article.

The star of today’s video is Bertrand Drouvot, who’s sporting a particularly impressive beard here. πŸ™‚

Cheers

Tim…

Video : TRUCATE TABLE … CASCADE

In today’s video we demonstrate the TRUNCATE TABLE … CASCADE feature added in Oracle 12.1.

The video is based on this article.

The star of today’s video is Øyvind Isene, who reluctantly took a break from his coffee to let me film this clip. πŸ™‚

Cheers

Tim…

Don’t be a moron. Install APEX in all your Oracle databases!

If you come from an Oracle background you know what Application Express (APEX) is. You know it’s an awesome low-code development environment, and you know it can turn DBAs and PL/SQL developers into productive members of society in no time at all. πŸ™‚

Even if you don’t want to use APEX, you should install APEX anyway, because it comes with some really handy PL/SQL packages. These come to mind.

APEX_WEB_SERVICE : I’ve written my own APIs in the past, but I never use them anymore. I recommend people make web service callouts from the database using the APEX_WEB_SERVICE package. You can use it for REST and SOAP calls, and it works equally well for JSON and XML web services.

APEX_JSON : Oracle included some JSON functionality in Oracle 12.1, but they only completed the implementation in Oracle 12.2. If you are working with a database version older than Oracle 12.2 and you care about JSON, you need the APEX_JSON package.

APEX_DATA_PARSER : This package allows you to convert CSV, JSON, XML and XLSX data into rows and columns. It’s really simple to use!

APEX_ZIP : You can do some simple gzip/gunzip stuff with the UTL_COMPRESS package, but the APEX_ZIP package allows you to zip up multiple files into an archive, or extract files from an existing archive.

APEX_MAIL : This package gives you an API over the APEX mail functionality. It’s more flexible than the UTL_MAIL package, and is much simpler than writing your own APIs using the UTL_SMTP package.

APEX_STRING : I always used to use the STRING_TO_TABLE and TABLE_TO_STRING functions in the APEX_UTIL package. Those have now been moved to the APEX_STRING package. You might prefer to use the SPLIT and JOIN functions instead. There is also a bunch of other string handling stuff worth checking out.

Every release brings more goodies for PL/SQL programmers, regardless of whether you want to use APEX or not!

So do yourself a favour. Install APEX into all your Oracle databases and make your PL/SQL developers happy.

Cheers

Tim…

PS. I don’t actually think you are a moron if you don’t install APEX. If you take the title of blog posts seriously, when they are clearly clickbait, maybe you are a moron…

Update: There seems to be a little confusion on some social media comments. Installing APEX into a database doesn’t mean APEX is available for people to develop applications. You can only use APEX proper if you have a gateway (ORDS, EPG, mod_plsql) fronting it. Without a gateway, APEX doesn’t work. Someone can’t take it upon themselves to run a gateway somewhere else if they don’t know all the passwords, so installing APEX and using it are not the same thing. Installing it does make the built-in packages available for normal PL/SQL development, which is really handy. Hence this post.

If you replace ‘enterprise application servers’ with ‘Oracle DB’ – does your message stay same? or different?

Yesterday I put out a post called Does anyone care about enterprise application servers anymore?

Before I posted it I got my boss to read it. This is how the Teams chat went.

  • Boss: It seems fine. Was there anything in particular you were not happy with?
  • Me: I guess I’m a little nervous about the post because the obvious comeback is, “Isn’t that true for the Oracle database too?”, and to some extent I would have to say yes*… I did consider adding a paragraph about that, but thought I would rather not.
  • Boss: Actually that is what I was thinking while reading it.

* In the context of the chat, “yes”, meant is was a valid question. Just not easy to answer.

Sure enough, not long after posting it I got a Twitter response from Timur Akhmadeev that said, “If you replace ‘enterprise application servers’ with ‘Oracle DB’ – does your message stay same? or different?” That is the title of this post. πŸ™‚

I’m going to attempt to explain my thoughts on this by going through the same talking points.

Full disclosure

I’m an Oracle fanboy. You know that. My take on this subject is bound to be biased because of my history.

I know I am mostly known for Oracle stuff, but in my current job I have to look after MySQL and SQL Server databases. I work on one project that uses PostgreSQL, which I’m really bad at. The company recently started using Snowflake, and the plan is to move all analytics and warehouse type stuff to that. So as much as I’m an Oracle fan, my work life is a lot more diverse than you might think.

Third-party application vendors

One of the major points in my original post was about the vendors of third-party products that got rid of enterprise application servers from their stack and replaced them with more streamlined alternatives. Do I see this reflected in the database space?

From my perspective no. We have some products that are 100% Oracle. Nothing we can do about that other than move to completely different products. Of those where the RDBMS engine is a choice, I don’t think we have one where they offer other RDBMS engines, but not Oracle. It seems Oracle is still a viable choice here. I’m not saying they are recommending it, but equally they are not advising against it.

There are some vendors where MySQL is not an option, but PostgreSQL is. Interesting…

Overall, we don’t see the same pressure from the vendors in the database space as we do in the enterprise application server space.

Containers

Similar to my comments about application servers, most RDBMS engines can work within containers, but should they? It’s fine for a demo or a small system. Are you happy about running a huge data warehouse as a monolith in a container though? I don’t think this is about Oracle or not. It’s something more fundamental than that.

Administration

There is definitely a big learning curve for Oracle. I could argue it’s probably harder to get good at Oracle administration than many other engines. I think you have to be really careful of the comparison though. Take my current company for example. The big and complicated stuff happens on Oracle, and the really simple stuff happens on SQL Server or MySQL. It’s unfair to compare the administration requirements because the use cases are so different.

I think the only way you can compare them is if someone has legitimately done the same types of projects on multiple engines and really understands them. I’m not that person, so I can’t make that comparison.

But what about product X?

As with the application server comment, if you have a vendor/product that demands a specific engine, you have no choice but to stick to it, or change vendor/product entirely.

For new products I would suggest you get some information about their current customer base and go with the majority/trend. If Oracle is an option, but 90% of their customers are not using Oracle, I would not knowingly make myself a minority without good reason.

Moving to the cloud

The cloud is the great leveller, and I think this is both good and bad for Oracle.

In the case of administration, cloud services can hide a lot of the complexity and make things much more generic. Take a look at RDS on AWS and you will see MySQL, SQL Server, PostgreSQL and Oracle all feel quite similar from a basic administration perspective. If people think Oracle is too complicated, take a look at RDS on AWS, or Autonomous Database on Oracle Cloud.

Where it becomes an issue for Oracle is as soon as the guts are hidden from us, how valuable is the Oracle IP? Imagine feature X is baked into the Oracle database, but requires 3 additional apps to achieve the same result with another RDBMS. On-prem this is a real pain, as it’s now my responsibility to get all this stuff working and keep it working. If a cloud service does all that for me, do I care if it is baked into the RDBMS or not? I don’t think I do. I just care about what features the service delivers and the price.

There is also the comparison of “built for the cloud” vs “monolith in the cloud”. I know this concerns a lot of people, but to be honest I don’t care. If the resulting service gives me what I need, how much do I care what’s under the hood? That’s the cloud provider’s problem, not mine.

Regardless of which engine you pick, I think you should assume you will be using a cloud database service unless there is a compelling reason not to, rather than the other way round.

RDBMS vs NoSQL?

I’m not going to get into the RDBMS vs NoSQL war because I think time has proved it to be redundant. I’ve been listening to this for about 15 years and it seems pretty clear to me the choice depends on the use case.

Some use cases fit really well with NoSQL and some don’t. There are also grey areas between where polyglot engines become interesting, but it all depends where your use case fits on the spectrum…

Imagine a shop that is mostly Oracle, but needs a JSON document store. Do they use a different engine like MongoDB, or just continue using Oracle and use SODA? The answer will be different depending on how you frame the question.

What’s the real problem with Oracle?

I don’t think Oracle’s problem is their tech, or even their prices. I speak to a lot of people and the number one problem they have with Oracle is the shady business practices. People don’t trust Oracle. That is not a good thing to hear from the customer base, especially when there are more alternatives than ever.

Oracle is in dire need of a Microsoft-style transformation, and that can only happen if the senior management at Oracle really want it to happen.

Is there a conclusion?

When you consider the context of the original blog post, not just the title, I don’t think you can just substitute “Oracle DB” for “enterprise application server” into the question. It’s too simplistic.

Cheers

Tim…

PS. The answer is, “it depends!” πŸ™‚