⚡️ Unlock High‑Performance Memoization for Any Argument Type

If you’ve been using functools.lru_cache you already know how sweet caching can be for pure functions.
But there’s a hidden snag: lru_cache only works with hash‑able arguments. As soon as you pass a list, dict, or a custom mutable object you’ll hit a TypeError.

Below is a drop‑in replacement that lets you cache any arguments – even unhashable ones – while still giving you the same LRU semantics, thread‑safety, and cache‑statistics you love.

🔍 How it works under the hood

1. Normalization – Every argument is transformed into an immutable, hash‑able representation.
* For built‑ins we recursively convert list → tuple, dict → frozenset of key/value pairs, set → frozenset.
* For custom objects we fall back to their id (the memory address) – safe because the object’s identity never changes during a single call.

2. Key construction – The normalized positional arguments and a sorted tuple of normalized keyword arguments are concatenated into a single tuple key.
This guarantees that (a=1, b=2) and (b=2, a=1) map to the same cache entry.

3. LRU mechanics – We reuse functools._lru_cache_wrapper logic via functools.lru_cache on an inner helper that receives the normalized key.
The wrapper handles the doubly‑linked list, hit/miss counters, and thread lock – we don’t have to reinvent that part.

4. Cache inspection – The wrapper still exposes .cache_info() and .cache_clear(), so you can monitor usage exactly like the std‑lib version.

💡 When to reach for this

* Functions that accept large data structures (e.g., pandas DataFrames, nested dicts) but are pure (no side‑effects).
* Recursive algorithms where the state is a mutable container – think DP on grids stored as lists of lists.
* APIs that receive JSON‑like payloads and would benefit from memoizing expensive validation or transformation steps.

🛠️ Full implementation

(continued below 👇)

import functools
import threading
from collections.abc import Mapping, Sequence, Set

def _make_hashable(obj):
    """Recursively turn mutable containers into immutable, hash‑able equivalents."""
    if isinstance(obj, (str, bytes, int, float, bool, type(None))):
        return obj                     # already hashable
    if isinstance(obj, Mapping):
        # Sort items to make order irrelevant
        return frozenset((k, _make_hashable(v)) for k, v in sorted(obj.items()))
    if isinstance(obj, (Set, frozenset)):
        return frozenset(_make_hashable(v) for v in sorted(obj))
    if isinstance(obj, Sequence):
        # Treat bytes as already handled; for lists/tuples, recurse
        return tuple(_make_hashable(v) for v in obj)
    # Fallback: use object identity – safe for the duration of a single call
    return id(obj)

def universal_lru_cache(maxsize=128, typed=False):
    """
    LRU cache decorator that works with any arguments, mutable or not.
    Mirrors functools.lru_cache signature.
    """
    def decorator(user_func):
        lock = threading.RLock()

        @functools.wraps(user_func)
        def wrapper(*args, **kwargs):
            # Normalise args to a hashable key
            key = (
                tuple(_make_hashable(arg) for arg in args),
                tuple(sorted((k, _make_hashable(v)) for k, v in kwargs.items()))
            )
            # Use the built‑in lru_cache on a thin inner function that receives the normalized key
            @functools.lru_cache(maxsize=maxsize, typed=typed)
            def _inner(normalised_key):
                # Unpack the original args back – we don't need them here because
                # the user function will be called with the *original* args/kwargs
                return user_func(*args, **kwargs)

            # The lock guards the first‑time creation of the inner function per wrapper
            with lock:
                return _inner(key)

        # expose the original lru_cache helpers for introspection
        wrapper.cache_info = lambda: wrapper.__wrapped__.cache_info()
        wrapper.cache_clear = lambda: wrapper.__wrapped__.cache_clear()
        return wrapper
    return decorator

⚙️ Example usage

@universal_lru_cache(maxsize=4)
def expensive_sum(matrix):
    """Pretend this is a heavy computation on a mutable list‑of‑lists."""
    total = 0
    for row in matrix:
        for val in row:
            total += val
    return total

# First call – cache miss
print(expensive_sum([[1, 2], [3, 4]]))   # → 10

# Second call with an *identical* mutable structure – cache hit!
print(expensive_sum([[1, 2], [3, 4]]))   # → 10 (instant)

# Different shape – cache miss
print(expensive_sum([[5, 5, 5], [5, 5, 5]]))  # → 30

print(expensive_sum.cache_info())
# CacheInfo(hits=1, misses=2, maxsize=4, currsize=2)

🔎 Why this works

* The normalization step guarantees that two semantically equal containers produce the exact same hash.
By turning a list into a tuple (and recursively doing the same for nested structures), we preserve the element order while gaining immutability.

* Sorting mappings removes the nondeterminism of dictionary insertion order, so {a:1, b:2} and {b:2, a:1} collide on purpose – they represent the same logical input.

* Using id as a fallback ensures that objects without a natural immutable representation (e.g., file handles, custom classes without __hash__) still get a stable key for the duration of that call. It’s safe because the cache key lives only as long as the function call; the object’s identity won’t change.

(continued below 👇)

Wrapping an inner function with functools.lru_cache lets us reuse the battle‑tested LRU implementation (including thread‑locking, eviction policy, and statistics) without re‑implementing those low‑level details. The outer wrapper simply translates between real arguments and the canonical* hashable key.

🚀 Takeaway checklist

* ✅ You can now memoize functions that accept any mutable data structure.
* ✅ Cache hits are deterministic – identical logical inputs, regardless of mutability, map to the same entry.
* ✅ No performance penalty beyond the one‑time normalization (which is cheap for most data shapes).
* ✅ Still get cache_info() and cache_clear() for introspection and control.

Give it a spin in your next data‑heavy project and watch those redundant calculations vanish! 🌟

🚀 One‑Sitting Python Project: “Silent Clipboard Historian”

Ever copy‑and‑paste a piece of code, a link, or a snippet and later wish you could scroll back to the exact thing you copied ? A clipboard logger gives you a personal “undo” for the clipboard, works offline, and fits in a single script you can run whenever you need it.

Why it works

1. pyperclip talks to the OS clipboard in a cross‑platform way (Windows, macOS, Linux).
2. The script polls the clipboard every few hundred milliseconds – fast enough for human speed but light on CPU.
3. By keeping the last seen value in memory we can detect changes and avoid logging the same text over and over.
4. Each new entry is stamped with datetime.utcnow() and appended to a JSON file. JSON is human‑readable, easy to load later, and works without a DB server.
5. A tiny CLI helper (--show N) lets you peek at the last N entries without opening the file manually.

All of this fits into ≈ 60 lines and needs only two third‑party packages.

Libraries / concepts you’ll touch

- pyperclip – clipboard access.
- json – serialising a list of dictionaries.
- datetime – UTC timestamps, ISO‑8601 format.
- argparse – a friendly command‑line interface.
- Polling loop – while True: … time.sleep(0.5) – a simple event‑driven pattern without extra dependencies.

Quick sketch of the core loop

import json, time, argparse, sys
from datetime import datetime
import pyperclip

LOGFILE = "clipboard_history.json"

def load_log():
    try:
        with open(LOGFILE, "r", encoding="utf-8") as f:
            return json.load(f)
    except (FileNotFoundError, json.JSONDecodeError):
        return []

def save_log(entry, history):
    history.append(entry)
    with open(LOGFILE, "w", encoding="utf-8") as f:
        json.dump(history, f, ensure_ascii=False, indent=2)

def main(show_last: int = None):
    if show_last:
        # ----- quick read‑only mode -----
        history = load_log()
        for rec in history[-show_last:]:
            print(f"[{rec['ts']}] {rec['text']!r}")
        return

    # ----- live‑logging mode -----
    history = load_log()
    last_clip = pyperclip.paste()  # initialise with current content
    print("📝 Clipboard historian started – press Ctrl‑C to stop.")
    try:
        while True:
            cur = pyperclip.paste()
            if cur != last_clip:
                entry = {"ts": datetime.utcnow().isoformat() + "Z", "text": cur}
                save_log(entry, history)
                print(f"🔖 Captured at {entry['ts']}")
                last_clip = cur
            time.sleep(0.5)          # half‑second pause → low CPU usage
    except KeyboardInterrupt:
        print("\n🛑 Stopped. History saved to", LOGFILE)

if __name__ == "__main__":
    parser = argparse.ArgumentParser(
        description="Silent Clipboard Historian – log everything you copy."
    )
    parser.add_argument(
        "-n", "--show", type=int, metavar="N",
        help="Print the last N clipboard entries and exit."
    )
    args = parser.parse_args()
    main(show_last=args.show)

What the snippet does

- load_log() safely opens the JSON file; if it doesn’t exist we start with an empty list.
- save_log() appends a new dict ({"ts": "...", "text": "…"}) and writes the whole list back – JSON’s indent=2 keeps the file readable.
- In live‑logging mode we keep last_clip in a local variable; every loop we compare the freshly‑pasted text. If it changed, we create a timestamp with datetime.utcnow().isoformat() + "Z" (the Z marks UTC) and dump it.
- The --show N flag bypasses the loop, loads the file, and prints the most recent N lines – perfect for quick look‑ups.

How to run it

(continued below 👇)

pip install pyperclip          # one‑off dependency
python clipboard_hist.py        # start logging
python clipboard_hist.py -n 5   # see the last 5 copies

Tips for extending (optional, after you’ve built it)

- Deduplication: store a hash of the text to avoid logging identical snippets that appear later.
- Desktop notifications: use plyer or win10toast to pop a tiny “saved” banner each time you copy.
- Search: add a --find <keyword> flag that filters entries containing the word.

Give it a try, tweak the polling interval if you feel adventurous, and enjoy having a personal “clipboard time‑machine” at your fingertips. Happy coding! 🎉

👋 Hey Python beginners!
Ever written a loop like this and wondered “How do I keep track of the position of each item?”

fruits = ["apple", "banana", "cherry"]
i = 0
for fruit in fruits:
    print(i, fruit)   # ❓ What’s i?
    i += 1

You’ve just invented a counter yourself – that works, but Python already gives us a cleaner, safer tool: enumerate.

What enumerate does, in plain words
- enumerate takes any iterable (like a list, tuple, or string).
- It adds a running number (starting at 0 by default) to each element and hands you a pair (index, value).
- Think of it as a “smart counter” that never forgets to increase, even if you break out of the loop early.

Why it’s better than a manual counter
1. Less code, less chance of mistakes – you don’t need to create, update, and remember a separate variable.
2. Works automatically with break or continue. If you skip an iteration, the counter stays correct.
3. Readability: anyone reading your code immediately sees you’re dealing with both an item and its position.

Using enumerate – the basics

fruits = ["apple", "banana", "cherry"]

for index, fruit in enumerate(fruits):
    print(f"Item {index}: {fruit}")

What happens?

1. enumerate(fruits) creates an iterator that yields (0, "apple"), then (1, "banana"), and so on.
2. The for loop unpacks each tuple into index and fruit.
3. Inside the body you can use both values freely.

Output

Item 0: apple
Item 1: banana
Item 2: cherry

Custom start number

Sometimes you want counting to start at 1 (human‑friendly) instead of 0. Just pass a second argument:

for pos, fruit in enumerate(fruits, start=1):
    print(f"{pos}. {fruit}")

Output

1. apple
2. banana
3. cherry

Why start=1 works: enumerate adds the offset you give it to the internal 0‑based counter before returning the pair.

Real‑world mini‑example: finding the first vowel in a word

word = "python"

for i, ch in enumerate(word):
    if ch.lower() in "aeiou":
        print(f"First vowel '{ch}' at position {i}")
        break
else:
    print("No vowel found")

- The loop checks each character.
- As soon as a vowel appears, we break out, and enumerate already knows the correct index (i).
- The else block runs only if the loop finishes without break, a handy pattern for “search‑and‑report” tasks.

TL;DR (the quick cheat‑sheet)

- enumerate(iterable, start=0) → gives you (index, value) pairs.
- Use it instead of manual counters for cleaner, safer loops.
- Adjust start when you need human‑readable numbering.

Give it a try in your next script – you’ll see how much smoother looping feels! 🚀

Happy coding! 🎉

🔧 Ever wanted to import Python code that lives somewhere other than a .py file on disk?

Maybe the source is stored in a database, fetched over the network, or generated on‑the‑fly.
The good news is that Python’s import machinery is entirely pluggable – you can teach the interpreter how to locate and load a module wherever you like.

Below is a self‑contained, intermediate‑to‑advanced example that shows you how to:

1. Create a custom finder that tells Python “I know how to locate this module”.
2. Build a loader that compiles the source code and injects the resulting module object.
3. Hook the finder into sys.meta_path so the interpreter automatically uses it.

> Why this works:
> The import system follows the PEP 302 protocol. When import foo runs, Python asks each entry in sys.meta_path (a list of finder objects) to produce a module spec (importlib.machinery.ModuleSpec). If a finder returns a spec, its associated loader is called to execute the module’s code. By providing our own finder/loader pair we can intercept the process at the exact point where the standard file‑system loader would normally kick in.

(continued below 👇)

# in_memory_importer.py
import sys
import types
import importlib.abc
import importlib.util
from hashlib import sha256

# ----------------------------------------------------------------------
# 1️⃣  A simple “virtual file system”: module name → source string
# ----------------------------------------------------------------------
VIRTUAL_MODULES = {
    "demo_greeting": """
def hello(name: str) -> str:
    return f"👋 Hello, {name}!"
""",
    "demo_math": """
# A tiny math helper – deliberately kept in a string
def add(a, b):
    return a + b

def mul(a, b):
    return a * b
""",
}

# ----------------------------------------------------------------------
# 2️⃣  Finder – decides whether it can handle a given fully‑qualified name
# ----------------------------------------------------------------------
class InMemoryFinder(importlib.abc.MetaPathFinder):
    def find_spec(self, fullname, path, target=None):
        # `fullname` is the absolute module name (e.g. "demo_greeting")
        if fullname in VIRTUAL_MODULES:
            # We create a spec that points to our custom loader
            return importlib.util.spec_from_loader(fullname, InMemoryLoader())
        # Returning None tells Python to try the next finder in the chain
        return None

# ----------------------------------------------------------------------
# 3️⃣  Loader – turns the source string into a real module object
# ----------------------------------------------------------------------
class InMemoryLoader(importlib.abc.Loader):
    def create_module(self, spec):
        """
        Optional hook. Returning None means “use the default module creation
        semantics” (i.e. a plain `types.ModuleType`). Keeping it simple.
        """
        return None

    def exec_module(self, module):
        """
        This is where the actual code execution happens.
        `module.__spec__` already contains the name we need.
        """
        source = VIRTUAL_MODULES[module.__spec__.name]
        # Compile the source to a code object – this gives us a proper
        # filename for tracebacks and enables `inspect.getsource`.
        code = compile(source, f"<in‑memory:{module.__spec__.name}>", "exec")
        # Execute the code in the module’s namespace.
        exec(code, module.__dict__)

        # Optional: give the module a stable __file__ attribute.
        # Some libraries (e.g. pathlib.Path(__file__).parent) rely on it.
        module.__file__ = f"<in‑memory:{module.__spec__.name}>"

        # Optional: compute a deterministic hash for caching tools.
        module.__hash__ = sha256(source.encode()).hexdigest()

# ----------------------------------------------------------------------
# 4️⃣  Register the finder – do this *once* at program start‑up
# ----------------------------------------------------------------------
sys.meta_path.insert(0, InMemoryFinder())

# ----------------------------------------------------------------------
# 5️⃣  Demo usage (you can delete this block in production)
# ----------------------------------------------------------------------
if __name__ == "__main__":
    import demo_greeting
    import demo_math

    print(demo_greeting.hello("Ada"))
    print("2 + 3 =", demo_math.add(2, 3))
    print("4 * 5 =", demo_math.mul(4, 5))

How the pieces interact

(continued below 👇)

- sys.meta_path is a list of finder objects consulted in order. By inserting our InMemoryFinder at position 0 we give it priority over the built‑in file‑system finder.
- find_spec receives the full module name. If the name exists in our VIRTUAL_MODULES dict we hand back a ModuleSpec that points to InMemoryLoader. Returning None is crucial; it tells the interpreter “I don’t know this one, try the next finder”.
- exec_module receives the module object that Python already created (or that create_module returned). We fetch the source, compile it (so the code object carries a sensible filename and line numbers), and exec it in the module’s dictionary. After exec finishes the module’s globals contain the functions, classes, etc., exactly as if the code had been loaded from a .py file.
- Why we compile: compiling gives us a code object that the interpreter can cache, introspect, and display proper tracebacks. It also lets tools like inspect.getsource work because we set module.__file__ to a recognizable pseudo‑filename.
- Thread safety: the import system already holds an internal lock (importlib._bootstrap._lock) while loading a module, so our loader does not need extra synchronization – the same module will never be executed twice concurrently.

When to reach for this pattern

- Configuration‑driven plugins stored in a database or remote service.
- On‑the‑fly generated code (e.g., JIT‑like scenarios where you synthesize Python source at runtime).
- Testing: you can inject mock modules without touching the filesystem, keeping test suites fast and isolated.
- Packaging tricks: bundle tiny helper modules inside a single binary (e.g., when using PyInstaller) and load them from a compressed archive.

💡 Quick sanity check: After running the script, try import demo_greeting; print(demo_greeting.__file__). You’ll see the pseudo‑filename <in‑memory:demo_greeting>, proving that the module thinks it lives somewhere virtual – yet it behaves exactly like any regular module.

Feel free to adapt the VIRTUAL_MODULES dict into a database query, an HTTP call, or any other data source. The import system is flexible enough that once you master the finder/loader contract, you can make Python import from anywhere. 🚀

🚀 One‑Sitting Python Project: “Smart Downloads Organizer”

Ever stare at a chaotic Downloads folder and wish the files would just file themselves? Build a tiny daemon that watches the folder, inspects each new item, and instantly ships it to a sensible sub‑directory (e.g., Images, Docs, Archives). All in under an hour, no GUI required.

Why it works

1. File‑system events are push‑based – the watchdog library creates an Observer that runs in a background thread and calls you only when something actually changes. No need for a busy‑wait loop that burns CPU.
2. MIME types give intent – Python’s built‑in mimetypes module maps a filename (or its contents via magic if you want extra accuracy) to a high‑level type like image/jpeg or application/pdf. Those top‑level strings (image, video, text, application) become the basis for your target folders.
3. Atomic moves keep things safe – shutil.move works across filesystems and automatically falls back to a copy‑+‑delete when a simple rename isn’t possible, so the script stays robust even if the user redirects downloads to a network drive.
4. Configurable rules = extensibility – By storing a tiny JSON/YAML mapping (.pdf → Documents, *.zip → Archives) you let the user tweak the behavior without touching code. The core loop stays unchanged.

What you’ll need

* watchdog – watches directories for create/modify/delete events.
* mimetypes (standard library) – guesses the file’s type from its name.
* shutil & pathlib – safe, cross‑platform path handling and moving.
* (Optional) python-magic – for content‑based MIME detection if you want extra accuracy.

High‑level flow

1. Load / create a mapping (type → folder).
2. Start a watchdog.Observer on the target directory.
3. For each FileCreatedEvent:
a. Wait a tiny moment (time.sleep(0.5)) to let the download finish.
b. Ask mimetypes.guess_type for the MIME string.
c. Derive the top‑level category (image, text, application, …).
d. Resolve the destination folder (fallback to Others).
e. shutil.move the file there, creating the folder if needed.
4. Keep the observer alive with a clean try/except KeyboardInterrupt block.

Code snapshot

(continued below 👇)

import json
import time
import shutil
from pathlib import Path
import mimetypes

from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler

# ---- Config ---------------------------------------------------------
# You can edit this file to change where each type lands.
CONFIG_PATH = Path.home() / ".downloads_organizer.json"
DEFAULT_MAP = {
    "image": "Images",
    "video": "Videos",
    "text": "Documents",
    "application": "Archives",   # e.g. zip, pdf, exe
    "audio": "Audio"
}
if CONFIG_PATH.exists():
    TYPE_MAP = json.loads(CONFIG_PATH.read_text())
else:
    TYPE_MAP = DEFAULT_MAP
    CONFIG_PATH.write_text(json.dumps(DEFAULT_MAP, indent=2))

# ---- Event handler --------------------------------------------------
class DownloadHandler(FileSystemEventHandler):
    def __init__(self, watch_path: Path):
        self.watch_path = watch_path

    def on_created(self, event):
        # We only care about files, not sub‑folders.
        if event.is_directory:
            return

        src = Path(event.src_path)

        # Give the OS a moment to finish writing the file.
        time.sleep(0.5)

        # Guess MIME type → "image/jpeg" → "image"
        mime, _ = mimetypes.guess_type(src.name)
        category = mime.split("/")[0] if mime else "unknown"

        # Resolve destination folder (fallback to "Others")
        folder_name = TYPE_MAP.get(category, "Others")
        dest_dir = self.watch_path / folder_name
        dest_dir.mkdir(exist_ok=True)

        dest = dest_dir / src.name
        try:
            shutil.move(str(src), str(dest))
            print(f"🗂️  Moved {src.name} → {dest_dir.name}/")
        except Exception as exc:
            print(f"⚠️  Failed to move {src.name}: {exc}")

# ---- Main loop ------------------------------------------------------
def main():
    downloads = Path.home() / "Downloads"
    handler = DownloadHandler(downloads)
    observer = Observer()
    observer.schedule(handler, str(downloads), recursive=False)
    observer.start()
    print(f"👀 Watching {downloads} … Press Ctrl+C to stop.")
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

if __name__ == "__main__":
    main()

What’s happening under the hood?

Observer.schedule registers handler for the Downloads* directory.
* When the OS creates a new file, on_created fires once – no polling.
* mimetypes.guess_type simply looks at the file extension; it’s fast and requires no extra binaries.
* shutil.move abstracts away os.rename vs. copy‑+‑delete, so you don’t need to think about cross‑disk moves.

⚡️ Take it further (optional challenges)

* Replace mimetypes with magic.from_file(..., mime=True) for content‑based detection.
Add a dry‑run* flag that only prints actions.
* Hook into the system notification center (plyer or notify2) to pop a toast each time a file lands.
Persist a log* (sqlite or plain CSV) of moved files for later analytics.

Give it a spin, watch your Downloads tidy itself, and feel the instant gratification of a script that actually does something useful every time you grab a file. Happy hacking! 🎉

👋 Hey Python newbies!
Ever needed to go through a list and also know which position you’re at?
You might have tried something like this:

for i in range(len(names)):
    print(i, names[i])

It works, but there’s a cleaner, safer way that Python gives you for free – enumerate(). Let’s unpack what it does and why it’s often the better choice.

🔎 What is enumerate()?
enumerate() takes any iterable (a list, tuple, string, etc.) and turns it into a sequence of pairs:

first element → the index (starting at 0 by default)
second element → the actual item from your original iterable

So instead of juggling two separate things (range and indexing), you get both in one tidy package.

🛠️ How to use it

names = ["Alice", "Bob", "Charlie"]

for idx, name in enumerate(names):
    print(f"#{idx + 1}: {name}")

What happens step‑by‑step?

1. enumerate(names) creates an iterator that yields (0, "Alice"), then (1, "Bob"), then (2, "Charlie").
2. The for loop unpacks each pair into idx and name.
3. Inside the block you can use both variables directly.

💡 Why is enumerate() nicer than range(len(...))?

✅ enumerate() — ❌ range(len(...))

Readability – “for each index and item” is obvious. — You have to remember to index the list manually.
Safety – Won’t raise an error if the list changes size during the loop. — If you accidentally change the list length, range may become out‑of‑sync.
Flexibility – You can start counting from any number (e.g., enumerate(seq, start=1)). — You’d need extra arithmetic (i+1).
Works with any iterable – even generators that don’t support len(). — Requires something that has a length (len()).

🔧 Quick variations

Start counting from 1 (often nicer for human‑readable output):

for pos, fruit in enumerate(["apple", "banana", "cherry"], start=1):
    print(f"{pos}. {fruit}")

Enumerate a string (each character gets an index):

for i, ch in enumerate("hello"):
    print(i, ch)

🚀 Mini challenge

Try printing the items of the list tasks = ["code", "debug", "test"] in reverse order while still showing the original index (0‑based). Hint: combine enumerate() with reversed().

tasks = ["code", "debug", "test"]

for idx, task in enumerate(reversed(tasks)):
    # your code here
    pass

Solution:

for idx, task in enumerate(reversed(tasks)):
    print(f"Original index {len(tasks)-1-idx}: {task}")

👍 Takeaway
Whenever you need both the position and the value while looping, reach for enumerate(). It makes your code clearer, less error‑prone, and works with any kind of iterable – a tiny tool that pays big dividends as you write more Python. Happy coding! 🚀

🛡️ Power‑up your network calls with a universal retry‑with‑backoff decorator

When you call external APIs, databases, or even local services, transient failures are the norm—not the exception. A bare‑bones try/except quickly becomes noisy and hard to maintain. Below is a single, reusable decorator that:

* Works for both sync and async functions (no duplicate code).
* Implements exponential back‑off with optional jitter (randomness) to avoid thundering‑herd effects.
* Lets you fine‑tune max attempts, base delay, and which exception types trigger a retry.
* Preserves the original function’s signature and metadata so tools like IDEs, help(), and type‑checkers stay happy.

👉 Why the pieces matter

1. functools.wraps – Copies __name__, __doc__, and the signature from the wrapped function to the wrapper. Without it, introspection (e.g. inspect.signature) would see the generic wrapper signature, breaking static analysis and docs.

2. Closure over is_async – We detect iscoroutinefunction once when the decorator is applied, not on every call. This avoids the overhead of re‑checking and lets us choose the correct await/sleep path.

3. Exponential back‑off formula

delay = base * (2 ** attempt)

Each retry doubles the wait time, giving the failing service time to recover.

4. Jitter (random.uniform(0, jitter)) – Adding a small random component prevents many clients from retrying exactly at the same moment, which could otherwise cause another spike of failures.

5. Separate sleep helpers – time.sleep blocks the thread; asyncio.sleep yields control back to the event loop. By abstracting them, the same retry logic works for both sync and async call‑sites.

6. Configurable exceptions – By default we retry on any Exception, but you can restrict it to, say, requests.exceptions.RequestException or a custom TransientError.

📦 The code

(continued below 👇)

import time
import random
import asyncio
import functools
import inspect
from typing import Callable, Type, Tuple, Union

def retry(
    *,
    attempts: int = 5,
    base_delay: float = 0.5,
    max_delay: float = 30.0,
    jitter: float = 0.1,
    exceptions: Union[Type[BaseException], Tuple[Type[BaseException], ...]] = Exception,
) -> Callable:
    """
    Decorator that retries the wrapped function with exponential back‑off.

    Parameters
    ----------
    attempts: int
        Maximum number of attempts (first try + retries).
    base_delay: float
        Starting delay in seconds.
    max_delay: float
        Upper bound for the back‑off delay.
    jitter: float
        Max random extra seconds added to each delay.
    exceptions: Exception or tuple
        Exception types that trigger a retry.
    """
    def decorator(func: Callable) -> Callable:
        is_async = asyncio.iscoroutinefunction(func)

        @functools.wraps(func)
        def sync_wrapper(*args, **kwargs):
            for attempt in range(attempts):
                try:
                    return func(*args, **kwargs)
                except exceptions as exc:
                    if attempt + 1 == attempts:
                        raise  # no more retries
                    delay = min(base_delay * (2 ** attempt), max_delay)
                    delay += random.uniform(0, jitter)
                    # Debug-friendly message – can be replaced with a logger
                    print(f"[retry] {func.__name__} failed ({exc}), retry {attempt+1}/{attempts} after {delay:.2f}s")
                    time.sleep(delay)

        @functools.wraps(func)
        async def async_wrapper(*args, **kwargs):
            for attempt in range(attempts):
                try:
                    return await func(*args, **kwargs)
                except exceptions as exc:
                    if attempt + 1 == attempts:
                        raise
                    delay = min(base_delay * (2 ** attempt), max_delay)
                    delay += random.uniform(0, jitter)
                    print(f"[retry] async {func.__name__} failed ({exc}), retry {attempt+1}/{attempts} after {delay:.2f}s")
                    await asyncio.sleep(delay)

        # Return the appropriate wrapper based on the original function type
        return async_wrapper if is_async else sync_wrapper

    return decorator

🚀 Quick demo

import requests
import asyncio

# ------------------- sync example -------------------
@retry(attempts=4, base_delay=1, jitter=0.2, exceptions=requests.exceptions.RequestException)
def fetch_status(url: str) -> int:
    """Fetch a URL and return HTTP status, retrying on network errors."""
    resp = requests.get(url, timeout=2)
    resp.raise_for_status()
    return resp.status_code

# ------------------- async example -------------------
@retry(attempts=3, base_delay=0.5, jitter=0.1, exceptions=ConnectionError)
async def async_fetch(url: str) -> str:
    """Async fetch that raises ConnectionError on failure."""
    reader, writer = await asyncio.open_connection(url, 80)
    # pretend we do something useful …
    writer.close()
    await writer.wait_closed()
    return "ok"

# Running the sync function
try:
    print(fetch_status("https://httpbin.org/status/503"))
except Exception as e:
    print("Final failure:", e)

# Running the async function
async def main():
    try:
        print(await async_fetch("example.com"))
    except Exception as e:
        print("Async final failure:", e)

asyncio.run(main())

What you’ll see

* On each failure the wrapper prints a concise retry log (replace print with logging in production).
* The delay grows 1 s → 2 s → 4 s (capped by max_delay if you set one).
* Jitter adds a small random offset, so two concurrent workers won’t line up perfectly.

(continued below 👇)

📌 Take‑away checklist

* Preserve signatures – always wrap with functools.wraps.
* Detect async once – saves a tiny but measurable cost inside the retry loop.
* Cap the back‑off – prevents runaway sleeps on stubborn services.
* Add jitter – a simple random.uniform can dramatically improve system stability under load.
* Make the decorator configurable – keep it reusable across projects rather than copy‑pasting ad‑hoc try/except blocks.

Drop this into a shared utilities module, and your codebase will instantly gain robust, self‑documenting retry logic without littering the business code with repetitive error handling. Happy coding! 🚀

🚀 One‑Sitting Python Project: “Smart Batch‑Renamer”

Ever stared at a folder full of screenshots, receipts, or logs and thought “There has to be a faster way to give them sensible names”? In a single sitting you can build a tiny CLI tool that scans a directory, extracts information with a regular‑expression, and renames every file in one go.

🎯 What the script does
1. Walks the target folder with pathlib.Path.rglob – a modern, object‑oriented wrapper around the filesystem that avoids the quirks of os.listdir.
2. Matches each filename against a user‑supplied pattern (e.g., IMG_(\d{4})(\d{2})(\d{2})_.*\.jpg). The capture groups let you rearrange parts of the original name.
3. Builds a new name using an f‑string template (e.g., f"{year}-{month}-{day}_{counter}.jpg"). Because f‑strings are evaluated at runtime, you can inject any variable you like – a counter, the current date, etc.
4. Renames safely with Path.rename. If a conflict appears, the script auto‑increments a numeric suffix so nothing gets overwritten.

> Why it works:
> * pathlib returns Path objects that behave like strings but also expose methods (suffix, stem, rename) – no need to concatenate strings manually.
> * re (the regex engine) returns a Match object; its .group() method extracts exactly the pieces you asked for, letting you transform the original filename without hard‑coding positions.
> * argparse gives a self‑documenting command line (the -p pattern, -t template, -d directory). Users can see --help and immediately understand the options.

🛠️ Core libraries / concepts
- pathlib – modern filesystem paths, cross‑platform.
- re – powerful pattern matching, essential for pulling out dates, IDs, etc.
- argparse – turn a script into a friendly CLI.
- itertools.count – an infinite counter for unique suffixes.

📦 Minimal skeleton (copy‑paste & run)

(continued below 👇)

#!/usr/bin/env python3
import argparse
import re
from pathlib import Path
from itertools import count

def parse_args():
    parser = argparse.ArgumentParser(
        description="Batch‑rename files using a regex pattern and a template."
    )
    parser.add_argument(
        "-d", "--directory", type=Path, default=Path.cwd(),
        help="Folder containing files to rename (default: current directory)."
    )
    parser.add_argument(
        "-p", "--pattern", required=True,
        help="Regex with capture groups to extract parts of the original name."
    )
    parser.add_argument(
        "-t", "--template", required=True,
        help=(
            "New name template using group names or numbers. "
            "Example: '{year}-{month}-{day}_{i}{ext}'"
        ),
    )
    parser.add_argument(
        "-r", "--recursive", action="store_true",
        help="Search sub‑folders as well."
    )
    return parser.parse_args()

def rename_files(dir_path: Path, regex: re.Pattern, tmpl: str, recursive: bool):
    files = dir_path.rglob("*") if recursive else dir_path.iterdir()
    counter = count(1)

    for file in files:
        if not file.is_file():
            continue
        m = regex.search(file.name)
        if not m:
            continue

        # Build a dict that can be used in the template
        groups = {str(i): v for i, v in enumerate(m.groups(), start=1)}
        groups.update(m.groupdict())
        groups["i"] = next(counter)          # fallback counter
        groups["ext"] = file.suffix
        groups["stem"] = file.stem

        new_name = tmpl.format(**groups)
        new_path = file.with_name(new_name)

        # Avoid overwriting existing files
        while new_path.exists():
            groups["i"] = next(counter)
            new_name = tmpl.format(**groups)
            new_path = file.with_name(new_name)

        print(f"Renaming: {file.name} → {new_path.name}")
        file.rename(new_path)

def main():
    args = parse_args()
    compiled = re.compile(args.pattern)
    rename_files(args.directory, compiled, args.template, args.recursive)

if __name__ == "__main__":
    main()

How to try it:
1. Save the script as renamer.py and chmod +x renamer.py.
2. Place a few test files (e.g., IMG_20230921_001.jpg, IMG_20230922_002.jpg).
3. Run:

./renamer.py -p "IMG_(\d{4})(\d{2})(\d{2})_.*\.jpg" \
             -t "{1}-{2}-{3}_{i}{ext}" -d ./photos

You’ll see the files renamed to 2023-09-21_1.jpg, 2023-09-22_2.jpg, …

🌟 Why you’ll love it
- Instant payoff – open a folder and instantly see a clean naming scheme.
- Fully customizable – change the regex or template to suit receipts, logs, audio files, etc.
- No external dependencies – everything lives in the Python standard library, so the script works on any machine with Python 3.8+.

Give it a spin, tweak the pattern, and you’ve got a reusable tool that saves minutes (or hours) every time you download a batch of files. Happy renaming! 🎉

Hey Python friends! 🌱
Today we’re diving into one of the simplest yet most powerful tricks for making your code readable and fun: f‑strings (short for “formatted string literals”).

Why do we need them?

When you want to show the value of a variable inside a piece of text, you have a few old‑school ways:

* Concatenation – "Hello, " + name + "!"
Works, but you have to remember to convert non‑strings with str() and it quickly becomes messy with many variables.

* % formatting – "Hello, %s!" % name
A relic from C‑style formatting; it’s harder to read when the format gets complex.

* str.format() – "Hello, {}!".format(name)
Better, but you still have to count placeholders or name them.

All of these are fine, but they force you to look away from the natural sentence you’re trying to build. That’s where f‑strings shine: they let you write the exact sentence you want, and Python fills in the blanks for you.

The magic behind f‑strings

1. Prefix the string with f (or F).

f"This is an f‑string"

2. Place any Python expression inside curly braces {}.
When the interpreter sees the string, it evaluates each expression and inserts its result right there.

3. The result is a new string (the original f‑string stays unchanged).

Because the evaluation happens at runtime, you can use any valid expression – not just variable names, but also calculations, function calls, or even conditional logic.

Simple example – greeting a user

name = "Ada"
age = 28

greeting = f"Hello, {name}! You are {age} years old."
print(greeting)

Output

Hello, Ada! You are 28 years old.

What’s happening?

- The f tells Python: “Hey, treat this as a template.”
- {name} is replaced by the current value of name.
- {age} is replaced by the current value of age.

No need for + or .format() – the text stays exactly as you’d read it.

Going a step further – expressions inside {}

width = 5
height = 3
area_msg = f"The area of a {width}×{height} rectangle is {width * height}."
print(area_msg)

Output

The area of a 5×3 rectangle is 15.

Here {width * height} is an expression. Python computes 5 * 3 and injects 15 right into the string.

Formatting numbers – keep things tidy

You can add a format specifier after a colon : inside the braces.

price = 9.99
tax = 0.075
total = price * (1 + tax)

msg = f"Subtotal: ${price:.2f}\nTax (7.5%): ${price*tax:.2f}\nTotal: ${total:.2f}"
print(msg)

Output

Subtotal: $9.99
Tax (7.5%): $0.75
Total: $10.74

Why does :.2f work?

- : introduces the format spec.
- .2f means “floating‑point number with 2 digits after the decimal”.
- Python applies that rule to the value before inserting it.

A quick do‑and‑don’t checklist

Do

- ✅ Use f‑strings for any string that includes variables or expressions.
- ✅ Keep the expression simple; complex logic belongs in a separate line.
- ✅ Add formatting specs (:.2f, :>10, etc.) for numbers, dates, or alignment.

Don’t

- ❌ Forget the leading f – without it the {} stay literal characters.
- ❌ Over‑nest braces ({{ and }} are only needed when you actually want a { or } in the final output).
- ❌ Use f‑strings for very large, multi‑megabyte texts where performance matters; plain concatenation or templates may be faster.

TL;DR – One‑liner cheat sheet

- Basic: f"Hello, {name}!"
- Expression: f"The sum is {a + b}"
- Number formatting: f"{value:.2f}" → two decimals
- Alignment: f"{text:>10}" → right‑align in a width of 10

Give it a try right now in your REPL:

(continued below 👇)

first = "Python"
second = "Rocks"
print(f"{first} {second}!".upper())

You’ll see:

PYTHON ROCKS!

Takeaway: f‑strings let you write what you mean instead of how to stitch it together. They make your code clearer, shorter, and more Pythonic – exactly the vibe we love as beginners.

Happy coding, and may your strings always be f‑abulous! 🚀

🌟 Python tip for absolute beginners: How to loop over a list and know the position of each item using enumerate

When you write a for‑loop you usually care about the value you’re iterating over:

fruits = ["apple", "banana", "cherry"]
for fruit in fruits:
    print(fruit)

But sometimes you also need to know where that value sits in the list – its index (0 for the first item, 1 for the second, …).

The “old‑school” way (and why it’s clunky)

for i in range(len(fruits)):
    print(i, fruits[i])

* Why it works: range(len(fruits)) creates a sequence of numbers from 0 up to the length of the list minus one. Inside the loop you use that number (i) to fetch the item with fruits[i].
* Why it feels awkward: you have to write the length of the list twice, you risk an “off‑by‑one” mistake, and the code mixes two different ideas (counting and accessing) in one line.

enumerate to the rescue

for index, fruit in enumerate(fruits):
    print(index, fruit)

What’s happening?

* enumerate(fruits) takes the original list and produces a new sequence of pairs.
* Each pair is a tuple: the first element is the current index, the second element is the original item.
* The for loop automatically unpacks that tuple into the variables index and fruit.

Because enumerate builds the pairs on the fly, it never creates a whole extra list in memory – it’s efficient even for very long sequences.

Starting the count at a different number

Sometimes you want the numbering to start at 1 (or any other number). enumerate lets you set a custom start:

for pos, fruit in enumerate(fruits, start=1):
    print(f"{pos}. {fruit}")

Output:

1. apple
2. banana
3. cherry

Why it works: the optional start argument tells enumerate what value to use for the first index. After that it simply adds 1 for each subsequent item.

Quick checklist – when to use enumerate

- You need the position of each item (e.g., to label output, to compare with another list, or to skip a specific index).
- You want cleaner code that avoids manual range(len(...)) gymnastics.
- You care about performance on large data – enumerate is lazy and memory‑friendly.

Mini‑exercise

Try this on your own:

words = ["hello", "world", "python", "rocks"]
for i, w in enumerate(words, start=5):
    print(f"Item {i}: {w.upper()}")

What do you see? Experiment by changing start and by removing it entirely. Notice how the output changes and how the code stays readable.

Takeaway: enumerate is a tiny built‑in tool that makes looping with indices both clear and safe. Keep it in your toolbox – you’ll reach for it every time you need “the item and its place” together. 🚀

🚀 One‑Sitting Python Project: “Smart Folder Organizer”

Ever looked at a messy Downloads folder and thought, “I wish my files would tidy themselves up”? In a single sitting you can build a tiny utility that scans any directory, decides where each file belongs (pictures, docs, archives, etc.) and moves it into a matching sub‑folder. The result is an instant declutter that you can run whenever you need – and the code itself is a great playground for the core ideas every new Python coder should master.

What you’ll learn (and why it matters)

Concept — Why it’s useful
**️⃣ pathlib.Path — Gives you an object‑oriented way to work with file system paths – clearer than the old os.path functions and works the same on Windows, macOS and Linux.
🔀 shutil.move — Handles moving files across directories (even across different drives) with a single call; you don’t have to reinvent copy‑and‑delete logic.
📂 Dictionaries — Perfect for mapping file extensions (e.g., .jpg) to destination folders (e.g., Images). This shows how data structures drive program flow.
🔁 Loops & conditionals — The backbone of any automation – you’ll iterate over every file, check its type, and decide what to do.
⚠️ Basic error handling — Using try/except protects you from permission errors or stray files that can’t be moved.

All of these are building blocks you’ll keep using in bigger projects, so mastering them now pays off later.

Quick sketch of the approach

1. Pick a folder – you can pass it as a command‑line argument or hard‑code a path for a quick test.
2. Define categories – a small dictionary where keys are tuples of extensions and values are the folder name you want them to end up in.
3. Iterate over every entry in the chosen directory with Path.iterdir().
4. Identify the file type by looking at Path.suffix.lower().
5. Create the target sub‑folder if it doesn’t already exist (Path.mkdir(exist_ok=True)).
6. Move the file with shutil.move().
7. Handle exceptions so the script never crashes on a single problematic file.

That’s it – roughly 30‑40 lines of code, and you have a reusable tool you can run manually or schedule with a cron job / Windows Task Scheduler.

Minimal but complete code snippet

import shutil
from pathlib import Path

# 1️⃣ Folder you want to clean up
SOURCE = Path.home() / "Downloads"          # change to any folder you like

# 2️⃣ Mapping of extensions → destination folder name
CATEGORIES = {
    ("jpg", "jpeg", "png", "gif", "bmp"): "Images",
    ("pdf", "doc", "docx", "txt", "odt"): "Documents",
    ("mp3", "wav", "flac"): "Audio",
    ("mp4", "mov", "avi", "mkv"): "Videos",
    ("zip", "tar", "gz", "rar"): "Archives",
}

def get_category(ext: str) -> str:
    """Return the folder name for a given file extension, or 'Other'."""
    for exts, folder in CATEGORIES.items():
        if ext in exts:
            return folder
    return "Other"

def organize_folder(folder: Path) -> None:
    """Walk through *folder* and move each file into its category."""
    if not folder.is_dir():
        raise NotADirectoryError(f"{folder} is not a directory")

    for item in folder.iterdir():
        # Skip sub‑folders – we only want to move files
        if item.is_dir():
            continue

        ext = item.suffix.lower().lstrip('.')   # e.g. 'jpg'
        target_folder = folder / get_category(ext)
        target_folder.mkdir(exist_ok=True)      # create if missing

        try:
            shutil.move(str(item), str(target_folder))
            print(f"✅ Moved {item.name} → {target_folder.name}/")
        except Exception as e:
            print(f"⚠️ Could not move {item.name}: {e}")

if __name__ == "__main__":
    organize_folder(SOURCE)

Why this works

(continued below 👇)