Oracle 8i | Oracle 9i | Oracle 10g | Oracle 11g | Miscellaneous | Oracle RAC | Oracle Apps | Linux

Oracle Database File System (DBFS) in Oracle Database 11g Release 2

Oracle has quite a long history with database file systems. The Oracle Internet File System (iFS) was released in the Oracle 8i days. This product was later renamed to Oracle Content Management SDK. The introduction of XML DB in Oracle 9i Release 2 brought with it a database file system accessible from HTTP, FTP and WebDAV for the storage of XML files. Now Oracle 11g Release 2 introduces DBFS, the Oracle Database File System.

DBFS creates a file system interface on top of database tables that store files as SecureFile LOBs. External access to the file system is via a client program (dbfs_client), which is only available for Linux and Solaris platforms. The client is already installed on the database server, but can also be installed on client machines, giving them access to the centralized database file system. On Linux platforms the dbfs_client can be used to mount the database file system on a regular mount point. This done using the "Filesystem in Userspace" (FUSE) project. This allows Linux machines to access DBFS like any other physical file system.

In this article I'll show the steps necessary to mount the DBFS on a Linux server. I'm not going to discuss the DBFS Content API, DBFS SecureFile Store or DBFS Hierarchical Store directly, although some of these are called by scripts used in this article.

FUSE Installation

In order to mount the DBFS we need to install the "Filesystem in Userspace" (FUSE) software. If you are not planning to mount the DBFS or you are running on an Non-Linux platform, this section is unnecessary.

Check to see if the "kernel-devel" package is installed.
rpm -q kernel-devel
kernel-devel-2.6.18-128.el5
#
If not, then install it using the following command.
# yum install kernel-devel
Download the FUSE 2.7.3 package from http://fuse.sourceforge.net/.

Determine the kernel directory.
# echo /usr/src/kernels/`uname -r`-`uname -p`
/usr/src/kernels/2.6.18-128.el5-x86_64
#
Install the FUSE package as the "root" user using the following commands, substituting your kernel directory.
# tar -xzvf fuse-2.7.3.tar.gz
# cd fuse-2.7.3
# ./configure --prefix=/usr --with-kernel=/usr/src/kernels/2.6.18-128.el5-x86_64
# make
# make install
# /sbin/depmod
# /sbin/modprobe fuse
# chmod 666 /dev/fuse
# echo "/sbin/modprobe fuse" >> /etc/rc.modules
# chmod 700 /etc/rc.modules

Creating a File System

First we must create a tablespace to hold the file system.
CONN / AS SYSDBA

CREATE TABLESPACE dbfs_ts
  DATAFILE '/u01/app/oracle/oradata/DB11G/dbfs01.dbf'
  SIZE 1M AUTOEXTEND ON NEXT 1M;
Next, we create a user, grant DBFS_ROLE to the user and make sure it has a quota on the tablespace. Trying to create a file system from the SYS user fails, so it must be done via another user.
CONN / AS SYSDBA

CREATE USER dbfs_user IDENTIFIED BY dbfs_user
  DEFAULT TABLESPACE dbfs_ts QUOTA UNLIMITED ON dbfs_ts;

GRANT CONNECT, CREATE TABLE, CREATE PROCEDURE TO dbfs_user;
GRANT dbfs_role TO dbfs_user;
Next we create the file system in tablespace by running the "dbfs_create_filesystem.sql" script as the test user. The script accepts two parameters identifying the tablespace and file system name.
cd $ORACLE_HOME/rdbms/admin
sqlplus dbfs_user/dbfs_user

SQL> @dbfs_create_filesystem.sql dbfs_ts staging_area
The script created a partitioned file system. Although Oracle consider this the best option from a performance and scalability perspective, it can have two drawbacks:
If these issues present a problem to you, you can create non-partitioned file systems using the "dbfs_create_filesystem_advanced.sql" script. In fact, the "dbfs_create_filesystem_advanced.sql" script is called by the "dbfs_create_filesystem.sql" script, which defaults many of the advanced parameters.

If we later wish to drop a file system, this can be done using the "dbfs_drop_filesystem.sql" script with the file system name.
cd $ORACLE_HOME/rdbms/admin
sqlplus test/test

SQL> @dbfs_drop_filesystem.sql staging_area

Mounting a File System

The dbfs_client tool is used to mount file systems on Linux servers. The usage is displayed if you call it without any parameters.
$ dbfs_client
usage: dbfs_client <db_user>@<db_server> [options] <mountpoint>
  db_user:              Name of Database user that owns DBFS content repository filesystem(s)
  db_server:            A valid connect string for Oracle database server
                        (for example, hrdb_host:1521/hrservice)
  mountpoint:           Path to mount Database File System(s)
                        All the file systems owned by the database user will be seen at the mountpoint.
DBFS options:
  -o direct_io          Bypass the Linux page cache. Gives much better performance for large files.
                        Programs in the file system cannot be executed with this option.
                        This option is recommended when DBFS is used as an ETL staging area.
  -o wallet             Run dbfs_client in background.
                        Wallet must be configured to get credentials.
  -o failover           dbfs_client fails over to surviving database instance with no data loss.
                        Some performance cost on writes, especially for small files.
  -o allow_root         Allows root access to the filesystem.
                        This option requires setting 'user_allow_other' parameter in '/etc/fuse.conf'.
  -o allow_other        Allows other users access to the file system.
                        This option requires setting 'user_allow_other' parameter in '/etc/fuse.conf'.
  -o rw                 Mount the filesystem read-write. [Default]
  -o ro                 Mount the filesystem read-only. Files cannot be modified.
  -o trace_file=STR     Tracing <filename> | 'syslog'
  -o trace_level=N      Trace Level: 1->DEBUG, 2->INFO, 3->WARNING, 4->ERROR, 5->CRITICAL [Default: 4]
  -h                    help
  -V                    version
[oracle@oel5-11gr2 admin]$
First we need to create a mount point with the necessary privileges as the "root" user.
# mkdir /mnt/dbfs
# chown oracle:oinstall /mnt/dbfs
The file system we've just created is mounted with the one of the following commands.
$ # Connection prompts for password and holds session.
$ dbfs_client dbfs_user@DB11G /mnt/dbfs

$ # Connection retrieves password from file and releases session.
$ nohup dbfs_client dbfs_user@DB11G /mnt/dbfs  < passwordfile.f &

$ # Connection authenticates using wallet and releases session.
$ nohup dbfs_client -o wallet /@DB11G_DBFS_USER /mnt/dbfs &
The wallet authentication is really the only sensible method to use as all other methods potentially expose the credentials. Creation of a wallet is discussed later.

Once mounted, the "staging_area" file system is now available for use.
$ ls -al /mnt/dbfs
total 8
drwxr-xr-x 3 root root    0 Jan  6 17:02 .
drwxr-xr-x 3 root root 4096 Jan  6 14:18 ..
drwxrwxrwx 3 root root    0 Jan  6 16:37 staging_area
$ ls -al /mnt/dbfs/staging_area
total 0
drwxrwxrwx 3 root   root     0 Jan  6 16:37 .
drwxr-xr-x 3 root   root     0 Jan  6 17:02 ..
drwxr-xr-x 7 root   root     0 Jan  6 14:00 .sfs
$
To unmount the file system issue the following command.
$ fusermount -u /mnt/dbfs

Wallet Creation

To create a wallet issue the following commands.
$ mkdir -p $HOME/oracle/wallet
$ mkstore -wrl $HOME/oracle/wallet -create
Add the following lines to the "$ORACLE_HOME/network/admin/sqlnet.ora" file.
WALLET_LOCATION = (SOURCE = (METHOD = FILE) (METHOD_DATA = (DIRECTORY = $HOME/oracle/wallet) ) ) 
SQLNET.WALLET_OVERRIDE = TRUE
Add the credentials to the wallet for a specific connect string. This could be the existing connect string (DB11G in mycase), but in this example I've used something a little more specific (DB11G_DBFS_USER).
mkstore -wrl $HOME/oracle/wallet -createCredential DB11G_DBFS_USER dbfs_user dbfs_user
Make sure the connect string is present in the "$ORACLE_HOME/network/admin/tnsnames.ora" file. In this case I just copied my existing connect string and gave it the new name.
DB11G_DBFS_USER =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = oel5-11gr2.localdomain)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = DB11G.WORLD)
    )
  )
The file system can now be mounted using the wallet.
$ nohup dbfs_client -o wallet /@DB11G_DBFS_USER /mnt/dbfs &

Using /etc/fstab

Warning: The documentation for this step is not complete and even when I try to fill in the gaps I'm not left with a working mount point. I've left this section in the document because I'm sure someone will very quickly inform me of my mistake. :)

Apart from having a familiar syntax, the use of the "/etc/fstab" file for mounting seems a little pointless since FUSE does not support automount, so you will still have to manually mount the file systems after a reboot. Actually this is fortunate as the automount would probably happen before the database was up, which would of course would be an issue.

To allow mount operations from the "/etc/fstab", issue the following commands as the "root" user.
# ln -s /u01/app/oracle/product/11.2.0/db_1/bin/dbfs_client /sbin/mount.dbfs
# groupadd fuse
# usermod -a -G fuse oracle
Add the following line into the "/etc/fstab" file.
/sbin/mount.dbfs#/@DB11G_DBFS_USER /mnt/dbfs fuse rw,user,noauto 0 0
Append the following lines into the "/etc/ld.so.conf" file.
/u01/app/oracle/product/11.2.0/db_1/lib
/usr/lib
Issue the following command. 
# ldconfig
The drive should now be mounted using the following command from the "oracle" user.
$ mount /mnt/dbfs
Note. At present this doesn't work for me. The command hangs, like it needs "nohup /mnt/dbfs &", but any access to the file system results in an "input/output error" message.
$ ls -al /mnt/dbfs
ls: /mnt/dbfs: Input/output error
$

dbfs_client Command Interface

As only Linux clients will be able to mount the file system, the majority of clients will access the file system via the dbfs_client command line, which provides a variety of UNIX-like file system command using the following syntax.
dbfs_client <db_user>@<db_serverr> --command <command> [switches] [arguments]
Commands include some typical UNIX file system commands like ls, cp, rm, mkdir. Arguments are typically file or directory names. The available switches are -a (all files), -l (long format) and -R (recursive).

Paths within the DBFS are referenced using the "dbfs:/file-system-name/" prefix. If this is not present, the path is assumed to be on the local filesystem of the machine running the client software.

Here are some examples of commands using the dbfs_client with the file system and wallet created earlier.
$ # Directory listing.
$ dbfs_client /@DB11G_DBFS_USER --command ls -a -l dbfs:/staging_area/
drwxr-xr-x              root            root               0    Jan 06 19:32    .sfs
$

$ # Make a directory and get directory listing.
$ dbfs_client /@DB11G_DBFS_USER --command mkdir dbfs:/staging_area/test_dir
$
$ dbfs_client /@DB11G_DBFS_USER --command ls -a -l dbfs:/staging_area/
drwx------            oracle        oinstall               0    Jan 07 11:41    dbfs:/staging_area/test_dir
drwxr-xr-x              root            root               0    Jan 06 19:32    .sfs
$

$ # Copy files to and from DBFS
$ dbfs_client /@DB11G_DBFS_USER --command cp /tmp/test.txt dbfs:/staging_area/test_dir/
/tmp/test.txt -> dbfs:/staging_area/test_dir/test.txt
$
$ dbfs_client /@DB11G_DBFS_USER --command cp dbfs:/staging_area/test_dir/test.txt /tmp/test_copy.txt 
dbfs:/staging_area/test_dir/test.txt -> /tmp/test_copy.txt
$

$ # Remove file
$ dbfs_client /@DB11G_DBFS_USER --command rm dbfs:/staging_area/test_dir/test.txt
Unlinking file dbfs:/staging_area/test_dir/test.txt
$
The command list is quite limited at the moment, but the message returned from unsupported commands suggests more are on their way.
$ dbfs_client /@DB11G_DBFS_USER --command rmdir dbfs:/staging_area/test_dir
rmdir is not supported right now
$
Switches must applied separately (-a -l), not grouped (-al) or the results are unpredictable. Also, wildcards do not work.

For more information see:
Hope this helps. Regards Tim...

Back to the Top.