Caching in Python
Caching is a technique in programming that improves performance by storing data that is frequently accessed in a temporary place. The cache is used to reduce access time, reduce system load and improve user experience. Here is a basic example of caching provided by Vivasoft:
import requests
cache = dict()
def get_data_from_server(url):
print("Fetching data from server...")
response: requests.get(url)
return response.text
def get_data(url):
if url not in cache:
cache[url] = get_data_from_server(url)
print("Getting data")
return cache[url]
get_data("python.org")
get_data("python.org")
Caching can be used at a software level and a hardware level. In Python, we are caching at a software level by using methods the language provides us.
When the data is ready to be accessed, the application looks for the results in the data structure also known as the cache. If the data is found it is returned straight from the cache. This is referred to as a cache hit. A cache miss is when the data is not present. The data is then added to the bottom of the cache and saved for future queries. Vytenis Kaubre at Oxylabs tells us "Different caching strategies can be devised based on specific spatial or temporal data access patterns.
Spatial caching is less popular in user-facing applications and more common in scientific or technical applications that deal with vast volumes of data and require high performance and computational power. It involves prefetching data that are spatially close (in the memory) to the data the application or system is currently processing. This idea exploits the fact that it’s more likely that a program or an application user will next demand the data units that are spatially close to the current demand. Therefore, prefetching can save time.
Temporal caching, a more common strategy, involves retaining data in the cache based on its frequency of use." There are five popular strategies for temporal caching FIFO(First In First Out), LIFO(Last In First Out), LRU(Least Recently Used), MRU(Most Recently Used) and LFU(Least Frequently Used).
There are 5 types of Python cache. Memory caching stores memory in a dict or library. It is fast and relies on available memory.
import time
from functools import lru_cache
# Simulating a slow function that fetches data from a database
@lru_cache(maxsize=128)
def get_data_from_database(id):
print('Fetching data from the database...')
time.sleep(2) # Simulating a delay
return f'Data for ID {id}'
# Usage
print(get_data_from_database(1))
print(get_data_from_database(2))
print(get_data_from_database(1))
In this example, we use the lru_cache decorator to create a memoized version of the get_data_from_database function. The lru_cache decorator automatically caches the results of function calls based on the arguments passed to the function. The maxsize parameter specifies the maximum number of results to cache. If the cache reaches this size, the least recently used result will be discarded to make room for new results.
When we call get_data_from_database(1) for the first time, it fetches the data from the database and stores it in the cache. On subsequent calls to get_data_from_database(1), it retrieves the data from the cache instead of executing the function again.
File-based Caching stores the data in files on a disk or distributed file system. This type can be slower but it can with larger datasets.
import os
import json
import time
CACHE_DIR = 'cache'
def create_cache_dir():
if not os.path.exists(CACHE_DIR):
os.makedirs(CACHE_DIR)
def get_cache_filepath(key):
return os.path.join(CACHE_DIR, f'{key}.json')
def write_to_cache(key, data):
create_cache_dir()
filepath = get_cache_filepath(key)
with open(filepath, 'w') as file:
json.dump(data, file)
def read_from_cache(key):
filepath = get_cache_filepath(key)
if os.path.exists(filepath):
with open(filepath, 'r') as file:
return json.load(file)
return None
def is_cache_valid(key, max_age):
filepath = get_cache_filepath(key)
if os.path.exists(filepath):
timestamp = os.path.getmtime(filepath)
age = time.time() - timestamp
return age <= max_age
return False
# Usage
def get_data_from_database(id):
print('Fetching data from the database...')
time.sleep(2) # Simulating a delay
return f'Data for ID {id}'
def get_data(id, cache_duration=60):
cache_key = f'data_{id}'
if is_cache_valid(cache_key, cache_duration):
print('Fetching data from cache...')
data = read_from_cache(cache_key)
else:
data = get_data_from_database(id)
print('Fetching data from the database...')
write_to_cache(cache_key, data)
return data
print(get_data(1))
print(get_data(2))
print(get_data(1))
In this example, we have a CACHE_DIR constant that specifies the directory where cache files will be stored. The create_cache_dir function ensures that the cache directory exists.
The get_cache_filepath function returns the file path for a given cache key. Each cache file is named based on the key and has a .json extension.
The write_to_cache function writes data to a cache file. It creates the cache directory if it doesn't exist and uses the json module to serialize the data as JSON.
The read_from_cache function reads data from a cache file. It checks if the cache file exists, and reads its contents using the json module, and returns the deserialized data.
The is_cache_valid function checks if a cache file is still valid based on its age. It compares the file's modification timestamp with the current time and returns True if the age is within the specified max_age.
The get_data function acts as a wrapper for fetching data. It takes an id parameter and an optional cache_duration parameter that specifies how long the data should be considered valid. It generates a cache key based on the id and checks if the data is already cached and still valid. If so, it reads the data from the cache file. Otherwise, it fetches the data from the source (simulated by get_data_from_database in this example), writes it to the cache, and returns the data.
Database caching can be slower but more persistent and can be used over multiple sessions. Here's an example of how to cache the results of a database query using Memcache:
import memcache
import sqlite3
# Connect to Memcache server
mc = memcache.Client(['localhost:11211'])
# Connect to SQLite database
conn = sqlite3.connect('your_database.db')
cursor = conn.cursor()
def get_data_from_database(id):
print('Fetching data from the database...')
cursor.execute('SELECT * FROM your_table WHERE id = ?', (id,))
data = cursor.fetchone()
return data
def get_data(id):
cache_key = f'data_{id}'
data = mc.get(cache_key)
if data is None:
data = get_data_from_database(id)
print('Fetching data from the database...')
mc.set(cache_key, data, time=60) # Cache data for 60 seconds
else:
print('Fetching data from cache...')
return data
# Usage
print(get_data(1))
print(get_data(2))
print(get_data(1))
# Close the database connection
conn.close()
In this example, we use the memcache library to connect to a Memcache server running on localhost at port 11211. You may need to adjust the connection details according to your Memcache setup.
The get_data_from_database function represents a database query that fetches data based on the given id. In this example, we assume a SQLite database, but you can adapt it to other database systems.
The get_data function acts as a wrapper for fetching data. It takes an id parameter and checks if the data is already cached in Memcache using the mc.get() method. If the data is present in the cache, it returns it. Otherwise, it fetches the data from the database using get_data_from_database, caches it in Memcache using the mc.set() method with a specified expiration time (in this case, 60 seconds), and returns the data.
Function caching uses the built-in functools.lru_cache decorator to prevent the need to re-execute the function.
import time
from functools import lru_cache
@lru_cache(maxsize=128)
def expensive_function(n):
print(f'Calculating result for {n}...')
time.sleep(2) # Simulating a delay
return n ** 2
# Usage
print(expensive_function(5))
print(expensive_function(5))
print(expensive_function(10))
print(expensive_function(10))
In this example, we have a function called expensive_function that performs some expensive computation. The lru_cache decorator from the functools module is used to create a memoized version of the function.
The maxsize parameter of the lru_cache decorator specifies the maximum number of results to cache. If the cache reaches this size, the least recently used result will be discarded to make room for new results.
When we call expensive_function(5) for the first time, it performs the computation and stores the result in the cache. On subsequent calls to expensive_function(5), it retrieves the result from the cache instead of recomputing it.
Similarly, when we call expensive_function(10) for the first time, it computes the result and stores it in the cache. On subsequent calls to expensive_function(10), it retrieves the result from the cache.
This caching mechanism helps improve the performance of the function by avoiding redundant computations for the same arguments.
Https caching only works for web apps. It stores http requests so the web server is not needed to be accessed so frequently.
from flask import Flask, request, make_response
app = Flask(__name__)
@app.route('/')
def index():
response = make_response('Hello, World!')
response.headers['Cache-Control'] = 'public, max-age=3600' # Cache response for 1 hour
return response
if __name__ == '__main__':
app.run()
In this example, we have a simple Flask web application with a single route (/) that returns the response "Hello, World!".
To enable HTTP caching, we set the Cache-Control header in the response object. The Cache-Control header specifies caching directives for the client and intermediaries (e.g., proxies). In this case, we set it to public, max-age=3600, which means the response can be cached by both the client and intermediaries for a maximum of 1 hour.
By setting appropriate caching headers, we allow the client and intermediaries to cache the response and serve it directly from their cache for subsequent requests within the specified max-age period.
The best practices for using caching are to choose the appropriate caching strategy, identify cache frequently accessed data, set appropriate cache expiration time, limit the size of the cache, use a consistent cache key format, monitor cache usage, and test performance impact consistently.
resources:
https://oxylabs.io/blog/python-cache-how-to-use-effectively
https://www.scaler.com/topics/python-cache/