Step 1: Remove Existing Packages

 

yum remove fuse fuse-s3fs


Step 2: Install dependency Packages

 

yum install openssl-devel gcc libstdc++-devel gcc-c++ fuse fuse-devel curl-devel libxml2-devel mailcap git automake


Step 3: Download and Compile Latest Fuse


cd /usr/src/
wget https://github.com/libfuse/libfuse/releases/download/fuse-3.0.1/fuse-3.0.1.tar.gz
tar xzf fuse-3.0.1.tar.gz
cd fuse-3.0.1
./configure --prefix=/usr/local
make && make install
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
ldconfig
modprobe fuse


Step 4: Download and Compile Latest S3FS


cd /usr/src/
wget https://github.com/s3fs-fuse/s3fs-fuse/archive/v1.82.tar.gz
tar xzf  v1.82.tar.gz
cd s3fs-fuse-1.82
./autogen.sh
./configure --prefix=/usr --with-openssl
make
make install


Step 5:Setup Access Key


To configure s3fs you need both access key and secret key of your s3 account

NOTE: Kindly replace the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with your actual key values separated by a colon


echo AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY > etc/passwd-s3fs
chmod 600 etc/passwd-s3fs



Step 6:Mount S3 Bucket on Linux


mkdir /tmp/cache
mkdir /s3mnt_pt
chmod 777 /tmp/cache /s3mnt_pt

s3fs BUCKET_NAME /s3mnt_pt -o url=http://FQDN_OF_S3_HOSTING_DOMAIN -o use_cache=/tmp/cache -o curldbg -o use_path_request_style -o allow_other



example for a "Personal Storage" s3 bucket hosted by Clouditalia

s3fs ps880392 /s3mnt_pt -o url=http://personal-storage.clouditalia.com -o use_cache=/tmp/cache -o curldbg -o use_path_request_style -o allow_other



note1: use_path_request_style option is mandatory since we are trying to connect to a non-amazon s3 implementation and the older path style is therefore needed

note2: curldbg option is not mandatory but it's useful for troubleshooting since it will give more output to the /var/log/messages file


use df -h to check, you shoud see s3fs filesystem mounted on /s3mnt_pt with 256T (terabytes!) of size


[root@ip-200-34 ~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   40G  7.1G   33G  18% /
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G     0  1.9G   0% /dev/shm
tmpfs                    1.9G  8.6M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1               1014M  189M  826M  19% /boot
/dev/mapper/centos-home   20G   99M   20G   1% /home
tmpfs                    380M     0  380M   0% /run/user/0
s3fs                     256T     0  256T   0% /s3mnt_pt



In case you want to access your s3 bucket just use normal cd , ls or ll command:


[root@ip-200-34 ~]# cd /s3mnt_pt/
[root@ip-200-34 s3mnt_pt]#


To unmount use one of the following:


fusermount -u /s3mnt_pt

or

umount /s3mnt_pt


Step 7:Make the mount persistent at boot


If you want to configure your system so that the S3 bucket is mounted when the system boots, then an entry can be added to /etc/fstab. Make sure you have the credentials stored in /etc/passwd-s3fs.


vi /etc/fstab


append the following line to the end of the file


s3fs#BUCKET_NAME /s3mnt_pt fuse _netdev,allow_other,use_path_request_style,url=http://FQDN_OF_S3_HOSTING_DOMAIN 0 0


example:


s3fs#ps880392 /s3mnt_pt fuse _netdev,allow_other,use_path_request_style,url=http://personal-storage.clouditalia.com 0 0


Once that entry is in place, then the bucket should be automatically mounted when the system boots.

Additionally, with the /etc/fstab entry in place, you can utilize mount and umount as needed:


umount /s3mnt_pt


mount /s3mnt_pt


umount /s3mnt_pt

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        21G  1.7G   18G   9% /
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G  8.4M  1.9G   1% /run
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           379M     0  379M   0% /run/user/0

mount /s3mnt_pt

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        21G  1.7G   18G   9% /
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G  8.4M  1.9G   1% /run
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           379M     0  379M   0% /run/user/0
s3fs            256T     0  256T   0% /mnt/s3mnt_pt



S3 Limitations - https://github.com/s3fs-fuse/s3fs-fuse

Random writes or appends to files require rewriting the entire file.
Metadata operations such as listing directories have poor performance due to network latency.
Eventual consistency can temporarily yield stale data(Amazon S3 Data Consistency Model).
No atomic renames of files or directories.
No coordination between multiple clients mounting the same bucket.
No hard links.


References

https://github.com/s3fs-fuse/s3fs-fuse - GitHub page for s3fs


Helpful links:


https://www.osris.org/documentation/s3fuse.html

https://kerneltalks.com/cloud-services/how-to-mount-s3-bucket-in-linux-server/

https://devops.profitbricks.com/tutorials/use-s3fs-fuse-to-access-s3-object-storage-on-centos-7/