Private Blob Storage With Minio

Jun 14th, 2019 - written by Kimserey with .

Minio is a open source distriubted storage. Few weeks ago, I showed how we could use it to store Microsoft Orleans actor states by implementing a grain storage backed with Minio. Today we will see how we can install two parts of Minio, the Minio server and the Minio client and we can update it when need be.

• Minio
• Minio Client
• Minio User

Minio

To install Minio, we follow the instructions from Minio to install the server:

1
2
3
cd /opt/minio
wget https://dl.minio.io/server/minio/release/linux-amd64/minio
chmod +x minio

We download the minio executable directly from the releases, and start it with server:

1
./minio server ./data

This will start Minio server and place all configurations under a single /data folder which will also hold the data.

To update Minio, we simply need to run the following and follow the CLI instructions:

1
2
3
minio update

You are already running the most recent version of ‘minio’.

In order to have Minio running as a service, we can create a systemd Unit under /etc/systemd/system/minio.service:

1
2
3
4
5
6
7
8
9
10
11
12
[Unit]
Description=Minio

[Service]
WorkingDirectory=/opt/minio
ExecStart=/opt/minio/minio server /opt/minio/data
SyslogIdentifier=minio
Restart=always
User=minio-user

[Install]
WantedBy=multi-user.target

And run the following:

1
2
3
systemctl enable minio
systemctl daemon-reload
systemctl start minio

Note that we also specified the user minio-user therefore we need to create it with:

1
sudo useradd -s /sbin/nologin minio-user

Now that we’ve seen how to install the server, we can install the client.

Minio Client

Minio also comes with a full featured client providing us a way to interact with our storage from command line with an implementation of the main shell commands.

1
2
3
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc -h

We configure a storage by using config with host add.

1
2
3
4
5
6
7
./mc config host add myminio http://[minio_ip]:9000 [access_key] [secret_key]

mc: Configuration written to `/kimserey/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/kimserey/.mc/share`.
mc: Initialized share uploads `/kimserey/.mc/share/uploads.json` file.
mc: Initialized share downloads `/kimserey/.mc/share/downloads.json` file.
Added `myminio` successfully.

We can then use all client commands specifying the host myminio:

1
./mc ls myminio

Similarly to minio, updating mc can be done by using update:

1
2
3
./mc update

You are already running the most recent version of ‘mc’.

Then if an update is available, we remove the old mc binary and download the new one.

Minio User Policy

In order to allow a user to administrate his own files, we provide a minio user with username and password. Together with the new user, we attribute a policy to the user allowing and denying certain action on the storage.

The list of all policies can be checked using the admin command with policy using the minio client:

1
2
3
4
5
./mc admin policy list myminio

readonly
readwrite
writeonly

To create our own policy we start by creating a mypolicy file containing the policy statements. The format follows s3 policies and get be generated directly from AWS policy generator.

As an example for our user, we will:

1. Allow listing of the files in a my-bucket bucket,
2. Allow Get and Put operations on the subpath my-bucket/configurations,
3. Deny any Put and Delete operations on the file my-bucket/files/important.txt,
4. Allow Get, Put and Delete operations on the rest of the files in my-bucket/files.

The order of the statemments matters in the fact that the deny statement affecting the file in my-bucket/files appears before the allow for the rest of the files. By default all operations are denied.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
tee ./policies/mypolicy << EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::my-bucket/*"
      ]
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::my-bucket/configurations/*"
      ]
    },
    {
      "Action": [
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Effect": "Deny",
      "Resource": [
        "arn:aws:s3:::my-bucket/files/important.txt"
      ]
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::my-bucket/files/*"
      ]
    }

  ]
}
EOF

We then use the file created to add the policy to minio using mc admin policy add:

1
./mc admin policy add myminio mypolicy ./policies/mypolicy

And lastly we can create a new user together with the permissions specified by the policy:

1
./mc admin user add myminio kimserey mypassword mypolicy

We will now be able to login to minio with access key as kimserey and secret key as mypassword. Once logged in we can see that only my-bucket is visible, we are able to list the documents in my-bucket and within those documents, we aren’t able to delete important.txt and we aren’t able to delete any files under configurations as specified by the policy.

External Sources

• Minio Quickstart
• Minio Client
• AWS policy generator

Conclusion

Today we saw how to install and update Minio server and Minio client. We also saw how we could setup Minio server to run as a daemon with systemd and we completed this post by looking at how to setup a new user with restricted permissions. Hope you like this post, see you next time!