Is there a program that emulate folder/file structure for databases in ubuntu?

Question

I have a database that has about 500,000 files. They're text files, and the folder structure divides them into groups of about 40 files each. I chose this structure because I wanted to be able to use a text editor to view each of the files. But I'd like to reduce the number of files I have, so backups go faster.

I'm looking for a program (or a database structure) that emulates a standard file structure, but is actually one file. My ideal structure would be to have something like a few .zip folders, but allow the files and folders in the .zip to be read, accessed, and written by file references from python and and a text editor (e.g., with a simple reference like "nano dataGroup1.zip/folderA/file.txt").

Is there such a program?

Edit: the backups I mentioned are a direct copies of the file structure. I copy and paste the entire folder to removable media.

Answer 1

you can use Fuse ( https://en.wikipedia.org/wiki/Filesystem_in_Userspace ) .

the project on github https://github.com/libfuse/libfuse .

This is a very mature project , so in Ubuntu you have already all packages .

Example :

1. create a zip

cd /tmp/
zip test_fuse_zip.zip /etc/printcap /etc/profile /etc/environment 

2. verify the content

unzip -l /tmp/test_fuse_zip.zip 

Archive:  test_fuse_zip.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
      135  2020-06-20 19:31   etc/printcap
      581  2018-04-09 07:10   etc/profile
      399  2020-06-18 23:07   etc/environment
---------                     -------
     1115                     3 files

3. create a mounting point

mkdir /tmp/my_zip_fs

4. mount the zip so it will appear has a file system

fuse-zip /tmp/test_fuse_zip.zip  /tmp/my_zip_fs/

5. do some modifications in folder

cd /tmp/my_zip_fs/

mkdir lib
mkdir usr
mkdir root
date > root/cur_date.txt

cd /tmp/

6. umount the zip

fusermount -u /tmp/my_zip_fs 

7. check the content

unzip -l /tmp/test_fuse_zip.zip 

Archive:  /tmp/test_fuse_zip.zip
Length      Date    Time    Name
---------  ---------- -----   ----
      135  2020-06-20 19:31   etc/printcap
      581  2018-04-09 07:10   etc/profile
      399  2020-06-18 23:07   etc/environment
        0  2020-06-21 13:43   lib/
        0  2020-06-21 13:43   usr/
        0  2020-06-21 13:44   root/
       29  2020-06-21 13:44   root/cur_date.txt
---------                     -------
 1144                     7 files