How Python enumerations make data configuration elegant. By Eric Burge, PhD | May, 2023

Python enums provide a more elegant solution for storing configuration information. Enums (short for enumeration) are essentially a way of defining a set of named constants. The best way to quickly understand enums is to adapt the machine learning configuration code from the previous section. You had the previous code:

# Configuration for RandomForestClassifier
N_ESTIMATORS_RANDOM_FOREST_CLASSIFIER = 100
MAX_DEPTH_RANDOM_FOREST_CLASSIFIER = 8
N_JOBS_RANDOM_FOREST_CLASSIFIER = 3
# Configuration for AdaBoostClassifier
N_ESTIMATORS_ADA_BOOST_CLASSIFIER = 50
LEARNING_RATE_ADA_BOOST_CLASSIFIER = 1.0

This can now be written as:

from enum import Enum
class RandomForest(Enum):
"""An Enum for tracking the configuration of random forest classifiers."""
N_ESTIMATORS = 100
MAX_DEPTH = 8
N_JOBS = 3
class AdaBoost(Enum):
"""An Enum for tracking the configuration of Ada boost classifiers."""
N_ESTIMATORS = 50
LEARNING_RATE = 1.0

some advantages

Let’s take a quick look at what benefits Python enums give us:

• Each parameter (eg, MAX_DEPTH) are now stored in a hierarchy within the model they use. This ensures that nothing is overwritten when more configuration code is introduced. So there is also no need for extremely long variable names.
• Various parameters used in RandomForestClassifier are now grouped in RandomForest Enum. Thus they can be iterated and collectively analyzed for type safety.
• Since Enums are classes, they can be docstrings As I have shown above. While it may not be strictly required for this example, in other examples it may be clear what the enumeration is referring to. Keeping it as a docstring that accompanies the class is much better, rather than a free-flowing comment. For one thing, automated documentation software will now pick up that the docstring is related to the enumeration. For free-flowing commentary, it would probably get lost.

If you now want to access the configuration code further down in the script, you can simply write:

from sklearn.ensemble import RandomForestClassifier
RandomForestClassifier(
n_estimators=RandomForest.N_ESTIMATORS.value,
max_depth=RandomForest.MAX_DEPTH.value,
n_jobs=RandomForest.N_JOBS.value
)

It looks clean and easy to read 😍

Some Python Features of Enums

Let’s illustrate some of the simple features of Python enumeration by looking at a toy example:

from enum import Enum
class HTTPStatusCodes(Enum):
"""An Enum that keeps track of status codes for HTTP(s) requests."""
OK = 200
CREATED = 201
BAD_REQUEST = 400
NOT_FOUND = 404
SERVER_ERROR = 500

Clearly, the information contained in the calculations is related; They are all about HTTP(S) status codes. Given this, you can now use the following simple code to extract the name and value:

print(HTTPStatusCodes.OK.name)
>>> OK
print(HTTPStatusCodes.OK.value)
>>> 200

Python enums also allow you to go backward: given that a status code value is 404You can find the status code name simply by typing:

print(HTTPStatusCodes(200).name)
>>> OK

You can work with name/value pairs in an Enum en masse, for example, by using a list builder list(),

print(list(HTTPStatusCodes))
>>> (
<HTTPStatusCodes.OK: 200>, 
<HTTPStatusCodes.CREATED: 201>, 
<HTTPStatusCodes.BAD_REQUEST: 400>, 
<HTTPStatusCodes.NOT_FOUND: 404>,
<HTTPStatusCodes.SERVER_ERROR: 500>
)

Finally, you can pickle and unpickle calculations in Python. To do this, just use pickle Model as you would with other familiar objects in Python:

from pickle import dumps, loads
print(HTTPStatusCodes is loads(dumps(HTTPStatusCodes)))
>>> True

This is especially important for the configuration code of machine learning models. In that case, there are also well known libraries like MLflow that elegantly save the configuration code for you.

sensitive configuration code

When using sensitive configuration code (such as passwords or access keys), you should never write them explicitly in code, They should be in a separate file that is ignored by whatever version control system you are using and imported into the script.

That doesn’t mean they can’t be stored in Enums within code. Enums are about organization, and censored information can be organized as well. As an example, here is an enumeration that represents a connection to a storage account in Microsoft Azure:

from enum import Enum
import os
class StorageAccount(Enum):
ACCOUNT_NAME = "my account name"
ACCESS_KEY = os.environ.get('ACCESS_KEY')
CONTAINER_NAME = "my container name"

right here StorageAccount fetching count ACCESS_KEY from an environment variable. This can be set for example, a .env file. Note that no sensitive information is exposed within the Python script. Nevertheless, all the information about the storage account is neatly organized in one enumeration.