How to use pydantic to read environment variables and secret files in Python
In our Python code, we often need to specify some sensitive information such as database username, password, API keys, JWT tokens, etc. We should not store any sensitive data as plain texts in our source code repository because they can get leaked easily. A common practice is to store the credentials as environmental variables or secret files on the machine on which the application is running. The machine is a generic concept and can be a bare-metal machine, a virtual machine, a docker container, a Cloud Run service, a Cloud Function, etc.
For simplicity, suppose that we have set up two environment variables for database username and password.
$ export DB_USERNAME=some_username
$ export DB_PASSWORD=some_password
To read environment variables, a common way is to use the
If the environment variable does not exist, using
os.environ will raise
KeyError. In this case, we can use
os.environ.get to avoid
KeyError. We can also pass a default value if the environment variable does not exist.
This classical way of dealing with environment variables is convenient when we just have one or two environment variables to deal with. However, it can be cumbersome and there would be a lot of duplicate code if we have more than a few environment variables. Besides, sometimes we will also have secret files for our code. In this example, the username can be set as an environment variable. However, the password should preferably be put in a secrete file, not as an environment variable. This is because the environment variable can be easily found by the