Oracle Technical Track

How to use mount-dbfs.sh to resolve "Device or resource busy"

4 min read

May 8, 2019 12:00:00 AM

I was not satisfied with using fuser -mv to identify processes causing DBFS failure to unmount and killing the processes haphazardly. Upon further investigation, mount-dbfs.sh can be used to kill processes causing "Device or resource busy" and unmount DBFS. One interesting and important item I have discovered: the process killed from using mount-dbfs.sh is ***not**** the same as shown from fuser. Stopping DBFS failed as shown: [code] oracle@host $ /u02/app/12.1.0/grid/crs/script/mount-dbfs.sh stop unmounting DBFS from /ggdata umounting the filesystem using '/bin/fusermount -u /ggdata' /bin/fusermount: failed to unmount /ggdata: Device or resource busy Stop - stopped, but still mounted, error oracle@host $ [/code] From another session, vi was used to open file from DBFS mount (ggdata). Notice the PID for vim is 38331. [code] root@host # fuser -mv /ggdata USER PID ACCESS COMMAND /ggdata: root kernel mount /ggdata ggsuser 2343 ..c.. bash ggsuser 38331 F.c.. vim root@host # exit [/code] Run mount-dbfs.sh clean in debug mode and notice: + /bin/kill -9 27532 [code] oracle@host $ /bin/bash -x /u02/app/12.1.0/grid/crs/script/mount-dbfs.sh clean + CONFIG=/etc/oracle/mount-dbfs.conf + VERSION=20160215 + '[' -r /etc/oracle/mount-dbfs.conf ']' + . /etc/oracle/mount-dbfs.conf ++ DBNAME=DBFS ++ MOUNT_POINT=/ggdata ++ DBFS_USER=dbfs_user ++ ORACLE_HOME=/u01/app/oracle/product/12.1.0/db_1 ++ GRID_HOME=/u01/app/12.1.0/grid ++ LOGGER_FACILITY=user ++ MOUNT_OPTIONS=allow_other,direct_io ++ PERL_ALARM_TIMEOUT=14 ++ DBFS_PASSWD=*** ++ DBFS_PWDFILE_BASE=/tmp/.dbfs-passwd.txt ++ WALLET=true ++ TNS_ADMIN=/u01/app/oracle/product/12.1.0/db_1/oracle/tnsadmin ++ DBFS_LOCAL_TNSALIAS=DBFS ++ IS_PDB=false ++ PDB=pdbXX ++ PDB_SERVICE=dbfspdb + CDB=DBFS ++ uname -s + UNAME_S=Linux + '[' Linux = Linux ']' + LINUX=1 + SOLARIS=0 + GREP=/bin/grep + AWK=/bin/awk + SED=/bin/sed + ECHO=/bin/echo + LOGGER='/bin/logger -t DBFS_/ggdata' + RMF='/bin/rm -f' + TOUCH=/bin/touch + CHMOD=/bin/chmod + PS=/bin/ps + SLEEP=/bin/sleep + KILL=/bin/kill + BASENAME=/bin/basename + STAT=/usr/bin/stat + ID=/usr/bin/id + WC=/usr/bin/wc + SRVCTL=/u01/app/oracle/product/12.1.0/db_1/bin/srvctl + DBFS_CLIENT=/u01/app/oracle/product/12.1.0/db_1/bin/dbfs_client + HN=/bin/hostname + PERL=/usr/bin/perl + MOUNT=/bin/mount + LANG=en_US.UTF-8 + NLS_LANG=American_America.AL32UTF8 + NUMACTL=/usr/bin/numactl + RPMCTL=/bin/rpm + '[' -z '' ']' + STATUS_TIMEOUT=0 + '[' 1 -eq 1 ']' + MOUNT=/bin/mount + XARGS='/usr/bin/xargs -r' + FUSERMOUNT=/bin/fusermount + LD_LIBRARY_PATH=/u01/app/oracle/product/12.1.0/db_1/lib:/lib64 + DBFS_PWDFILE=/tmp/.dbfs-passwd.txt.39786 + export ORACLE_HOME LD_LIBRARY_PATH TNS_ADMIN + export STAT MOUNT_POINT PERL_ALARM_TIMEOUT SOLARIS LINUX + export PATH=/u01/app/oracle/product/12.1.0/db_1/bin:/u01/app/oracle/product/12.1.0/db_1/bin:/usr/java/jdk1.8.0_65/bin:/usr/java/jdk1.8.0_65/jre/bin:/usr/java/jdk1.8.0_65/jre/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/oracle/.local/bin:/home/oracle/bin + PATH=/u01/app/oracle/product/12.1.0/db_1/bin:/u01/app/oracle/product/12.1.0/db_1/bin:/usr/java/jdk1.8.0_65/bin:/usr/java/jdk1.8.0_65/jre/bin:/usr/java/jdk1.8.0_65/jre/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/oracle/.local/bin:/home/oracle/bin + export STATUS_TIMEOUT + export LANG NLS_LANG ++ /usr/bin/id -u + '[' 1500 -eq 0 ']' + SCRIPTPATH=/u02/app/12.1.0/grid/crs/script/mount-dbfs.sh ++ /bin/basename /u02/app/12.1.0/grid/crs/script/mount-dbfs.sh + SCRIPTNAME=mount-dbfs.sh + /bin/echo /u02/app/12.1.0/grid/crs/script/mount-dbfs.sh + /bin/grep '^/' + '[' 0 -ne 0 ']' + cd /tmp + case "$1" in + logit info 'cleaning up DBFS nicely using (fusermount -u|umount)' + type=info + msg='cleaning up DBFS nicely using (fusermount -u|umount)' + '[' info = info ']' + /bin/echo cleaning up DBFS nicely using '(fusermount' '-u|umount)' cleaning up DBFS nicely using (fusermount -u|umount) + /bin/logger -t DBFS_/ggdata -p user.info 'cleaning up DBFS nicely using (fusermount -u|umount)' + '[' 1 -eq 1 ']' + /bin/fusermount -u /ggdata /bin/fusermount: failed to unmount /ggdata: Device or resource busy + /bin/sleep 1 + FORCE_CLEANUP=0 + '[' 0 -gt 1 ']' + /u02/app/12.1.0/grid/crs/script/mount-dbfs.sh status + '[' 0 -eq 0 ']' + FORCE_CLEANUP=1 + '[' 1 -eq 1 ']' + logit error 'tried (fusermount -u|umount), still mounted, now cleaning with (fusermount -u -z|umount -f) and kill' + type=error + msg='tried (fusermount -u|umount), still mounted, now cleaning with (fusermount -u -z|umount -f) and kill' + '[' error = info ']' + '[' error = error ']' + /bin/echo tried '(fusermount' '-u|umount),' still mounted, now cleaning with '(fusermount' -u '-z|umount' '-f)' and kill tried (fusermount -u|umount), still mounted, now cleaning with (fusermount -u -z|umount -f) and kill + /bin/logger -t DBFS_/ggdata -p user.error 'tried (fusermount -u|umount), still mounted, now cleaning with (fusermount -u -z|umount -f) and kill' + '[' 1 -eq 1 ']' + /bin/fusermount -u -z /ggdata + '[' 1 -eq 1 ']' ++ /bin/ps -ef ++ /bin/grep -w /ggdata ++ /bin/grep dbfs_client ++ /bin/grep -v grep ++ /bin/awk '{print $2}' + PIDS=27532 + '[' -n 27532 ']' -------------------------------------------------------------------------------- + /bin/kill -9 27532 -------------------------------------------------------------------------------- ++ /bin/ps -ef ++ /bin/grep -w /ggdata ++ /bin/grep mount.dbfs ++ /bin/grep -v grep ++ /bin/awk '{print $2}' + PIDS= + '[' -n '' ']' + exit 1 [/code] Here are the parameters for mount-dbfs.sh: [code] oracle@host $ /u02/app/12.1.0/grid/crs/script/mount-dbfs.sh -h Usage: mount-dbfs.sh { start | stop | check | status | restart | clean | abort | version } oracle@host $ [/code] In conclusion, it's much better to use the script for what it's designed for versus manually having to identify and kill processes.

On this page

Ready to unlock value from your data?

With Pythian, you can accomplish your data transformation goals and more.

Speak with Pythian consultants now →

What We Do

How to use mount-dbfs.sh to resolve "Device or resource busy"

Share this

Share this

More resources

How To Resolve Database Software Install To Incorrect Location

How to Automate Killing Inactive Sessions with Resource Manager

How to Find Cluster Resources Where Target!=State

Ready to unlock value from your data?