Monumento Gcloud Storage

Python helpers for Google Cloud Storage: credentials, buckets, and objects.

• Monumento Gcloud Storage
• Requirements
• Install
• Handling credentials
  • Local mode - Using a JSON file path
    • Hardcoding the path
    • Using an environment variable
  • Production mode - Converting the JSON file to a string
• Usage
  • Creating a bucket
  • Uploading a file
  • Listing files
    • Listing files using prefix
  • Download a file
  • Deleting a file
  • Emptying a bucket
  • Deleting a bucket
• Contributing
• Security
• Author

Requirements

• Google cloud account
• Activated google cloud storage API
• The credentials associated with the use of the service, generally a JSON file.

Install

To install the project, clone it, and run:

uv sync

Optional extras install Pillow (images) and pymongo (mongo). They are not part of the public API (GCSManager, ServiceAccountCredentials):

uv sync --extra images --extra mongo

Handling credentials

To use the google cloud storage service, you need to provide the credentials associated with the service. The most common and secure way to provide the credentials is using a JSON file. While in development it is ok to have the credentials in a file, in production it is recommended to use environment variables.

Local mode - Using a JSON file path

You can use the application with a path to a JSON file containing the credentials in different manners.

Hardcoding the path

You can hardcode the path to the JSON file in the code.

from monugstore import GCSManager

PATH_TO_CREDENTIALS = "/path/to/credentials.json"
gcs_manager = GCSManager.from_json_file(PATH_TO_CREDENTIALS)

Using an environment variable

You can set an environment variable with the path to the JSON file.

export PATH_TO_CREDENTIALS=/path/to/credentials.json

Or create a .env file with the following content:

PATH_TO_CREDENTIALS=/path/to/credentials.json

And read the environment variable in the code.

import os
from monugstore import GCSManager

PATH_TO_CREDENTIALS = os.getenv("PATH_TO_CREDENTIALS")
gcs_manager = GCSManager.from_json_file(PATH_TO_CREDENTIALS)

Production mode - Converting the JSON file to a string

We have included a command to deal with the issue of storing the json file in production mode, where it is not recommended to store the file in the repository. To convert the JSON file to a string, you can run the following command:

mgs-dump-key /path/to/credentials.json

You will receive a string that you can use in the code with environment variables. In your .env file, you can add the following line:

CREDENTIAL_STRING=<string>

and in the code, you can use the following code:

import os
from monugstore import GCSManager

CREDENTIAL_STRING = os.getenv("CREDENTIAL_STRING")
gcs_manager = GCSManager.from_json_string(CREDENTIAL_STRING)

Alternatively, pass the environment variable name to from_env:

from monugstore import GCSManager

gcs_manager = GCSManager.from_env("CREDENTIAL_STRING")

Usage

After setting up the credentials, you can use the GCSManager class to interact with the google cloud storage service. Without the steps mentioned above, you will not be able to use the service. Let's say that you are using a hardcoded path to the JSON file.

from monugstore import GCSManager

PATH_TO_CREDENTIALS = "/path/to/credentials.json"
gcs_manager = GCSManager.from_json_file(PATH_TO_CREDENTIALS)

Creating a bucket

You have an instance of the GCSManager class, and you can use it to create a bucket. New buckets are private by default. Pass public=True to make the bucket and future objects public.

bucket_name = "my-bucket"
gcs_manager.create_bucket(bucket_name=bucket_name)
gcs_manager.create_bucket(bucket_name=bucket_name, public=True)

Uploading a file

You can upload a file to the bucket using the upload_file method. It will return a public url associated with the blob on the bucket.

bucket_name = "my-bucket"
path_to_file = "path/to/file.jpg"
destination_blob_name = "file.jpg"
prefix = "uploads"

public_path = gcs_manager.upload_file(
    bucket_name,
    path_to_file,
    destination_blob_name,
    prefix=prefix,
)

Listing files

To list files on a bucket you can run the following method:

bucket_name = "my-bucket"

file_names = gcs_manager.list_files(bucket_name)

Listing files using prefix

You can also list files using a prefix:

bucket_name = "my-bucket"
prefix = "path/to/files"

file_names = gcs_manager.list_files(bucket_name, prefix)

Download a file

You can download a file from the bucket using the download_file method.

bucket_name = "my-bucket"
destination_path = "path/to/destination"

gcs_manager.download_file(bucket_name, "file.jpg", destination_path)

Deleting a file

You can delete a file from the bucket using the delete_file method.

bucket_name = "my-bucket"
file_name = "file.jpg"

gcs_manager.delete_file(bucket_name, file_name)

Emptying a bucket

You can delete every object in a bucket with delete_all_files. The bucket must exist.

gcs_manager.delete_all_files("my-bucket")

Deleting a bucket

You can delete a bucket using the delete_bucket method.

bucket_name = "my-bucket"

gcs_manager.delete_bucket(bucket_name)

Contributing

See CONTRIBUTING.md. Notable changes are listed in CHANGELOG.md.

Security

See SECURITY.md to report a vulnerability.

Author

This project was created by Pedro Cavalcanti You can contact at:

• Linkedin
• Email: pedro@monumentosoftware.com.br