Reddit Programming
200 subscribers
1.22K photos
126K links
I will send you newest post from subreddit /r/programming
Download Telegram
The Contract Your Test Didn’t Mean to Sign
https://www.reddit.com/r/programming/comments/1sugrij/the_contract_your_test_didnt_mean_to_sign/

<!-- SC_OFF -->A while ago I posted about the gap between what e2e tests appear to prove and what they actually check. The discussion around that made me think more about the part I may not have understood well enough: tests do not just check software. They write contracts for what the system must continue to preserve. And sometimes, without noticing, they write a bigger contract than the promise needed. A clean test can still make the wrong commitment, if it ties the system to a surface that changes faster than the behavior it was meant to protect. It will still become brittle. That is the contract your test did not mean to sign. Small example: promise: a business party can be created contract actually encoded in a UIbasede2e test: PartyList -> click "Add party button" -> PartyModal -> click "Business tab" -> Fill "party name" with "Acme Inc." -> click "submit" -> new party row with "Acme Inc." appears Same promise space, UI-agonistic contract: parties -> addBusiness 'Acme Inc.' parties -> get 'Acme Inc.' -> exists Neither version is universally better. They just commit the system to different things. The problem starts when the test claims to protect one promise, but quietly depends on a surface that changes for different reasons. That is where a lot of hidden brittleness enters test suites. Once the promise and the contract move at the same pace, the whole suite gets easier to reason about: a UI contract changes when UI behavior changes an application contract changes when the capability changes mechanical failures are easier to locate it becomes clearer when a lower-level check creates more churn than the promise is worth and if a test is truly UI-scope, it is worth asking whether e2e is the right place for it, or whether a smaller UI/component test would give faster, more focused feedback. I wrote the longer version in the linked blog post if you find this discussion interesting. Appreciate any feedback, and happy to partake in discussions! :) <!-- SC_ON --> submitted by /u/TranslatorRude4917 (https://www.reddit.com/user/TranslatorRude4917)
[link] (https://www.abelenekes.com/p/the-contract-your-test-didn-t-mean-to-sign) [comments] (https://www.reddit.com/r/programming/comments/1sugrij/the_contract_your_test_didnt_mean_to_sign/)
How I Built an Automated JS/TS Repository Analyzer for the Silverfish IDP
https://www.reddit.com/r/programming/comments/1suhkea/how_i_built_an_automated_jsts_repository_analyzer/

<!-- SC_OFF -->TL;DR I built the JavaScript/TypeScript analysis engine for the Silverfish IDP, an Internal Developer Portal that automatically detects packaging tools, identifies component types, and extracts complete dependency graphs from repos. It handles monorepos, multiple lock file formats, and mixed JS/TS codebases—all whilst minimising assumptions about the expected repo structure. The Problem The aim of the Silverfish IDP is to help individual developers and engineering teams understand their entire codebase. But when you have hundreds of repositories spanning multiple languages, frameworks, and tools, how do you automatically make sense of it all? For JavaScript and TypeScript repos specifically, the challenge is significant: every repo is different. Some use Yarn, others npm or pnpm. Some have monorepos with nested package.json files. Some mix JavaScript and TypeScript. Some have multiple lock files checked in (a real mess). And some don't have lock files at all. I needed an analyzer that could handle all these cases automatically with no manual configuration, no "please tell us which package manager you use" questions. Just point it at a repo and get back structured metadata about components, dependencies, and versions. Step 1: Detect the Packaging Tool The naive approach: Check if yarn.lock exists → use Yarn. Check if package-lock.json exists → use npm. Reality is messier: // Priority order matters 1. Check packageManager field in package.json ("yarn@4.1.0") 2. Look for lock files (yarn.lock, pnpm-lock.yaml, package-lock.json, bun.lock) 3. Check config files (.yarnrc.yml, pnpm-workspace.yaml) 4. Default to npm The packageManager field was the key insight—it's set by corepack and is the source of truth. If it says Yarn, it's Yarn, even if npm somehow created a lock file too. I also had to handle conflicts: I found real repos with both yarn.lock and package-lock.json checked in. My solution? Detect all of them, report the conflict, and parse only the highest-priority one. C# public static async Task DetectAsync( IReadOnlyCollection repoPaths, Func> readFileContentAsync) { // 1. Check packageManager field first var fromPackageManager = await TryDetectFromPackageManagerFieldAsync(...); if (fromPackageManager is not null) return fromPackageManager; // 2. Check lock files var fromLockFile = TryDetectFromLockFiles(...); if (fromLockFile is not null) return ...; // 3. Check config files var fromConfigFile = TryDetectFromConfigFiles(...); if (fromConfigFile is not null) return ...; // 4. Default to npm return new(PackagingTool.Npm, true); } Result: (PackagingTool.Yarn, LockFileNeedsGenerating: false) or similar. Step 2: Identify Components and Their Type Each package.json is a component. But what kind? And what does it do? I classified each one into: Package (capable of being published to npm), Library (internal or private), and determined usage: Frontend, Backend, Fullstack, or Unknown. The key was looking at dependencies: C# static readonly HashSet FrontendSignals = new() { "react", "vue", "@angular/core", "svelte", "react-router", "redux", ... }; static readonly HashSet BackendSignals = new() { "express", "koa", "mongoose", "pg", "apollo-server", "prisma", ... }; // If a package depends on react + express = fullstack // If only react = frontend // If only express = backend I also extracted language info: C# // Pure JS? Check for no TypeScript signals // TypeScript? Look for typescript pkg + /* // Mixed? Has flow-bin + typescript OR tsconfig.json's allowJs = true And pulled in version constraints: C# // Node version: from engines.node in package.json or .nvmrc file // TS version: from devDependencies // ECMAScript target: from tsconfig.json compilerOptions Result: A JsComponent record with all metadata attached—used by Silverfish's
dashboard to display component details instantly. Step 3: Parse Lock Files (The Hard Part) This was the gnarly part. Four different formats, each with quirks. Yarn Lock (v1 Classic) Looks like TOML with nested dependency lists: Code "@pkgjs/parseargs@^0.11.0": version "0.11.0" resolved "https://registry.npmjs.org/..." dependencies: package-json "^6.0.0" I wrote a line-by-line parser. The trick: track indentation to know when you're inside a package block vs. dependency list. npm package-lock.json Flat JSON structure (v2/v3): JSON { "packages": { "node_modules/lodash": { "version": "4.17.21", "dependencies": { ... } } } } Easier to parse with JsonDocument, but the key names have node_modules/ prefixes that need stripping. pnpm-lock.yaml YAML with name@version keys: YAML packages: /lodash/4.17.21: version: 4.17.21 dependencies: react: 18.2.0 I treated this as mostly line-based text parsing since I didn't want to add a full YAML dependency. Works for the common cases. Bun Lock JSONC format with array-based entries. Least common, so I parse it but mark binary bun.lockb files as unparseable. Step 4: Resolve Dependencies Once I had a parsed lock file, I needed to extract: Local dependencies (internal workspace packages like u/company/shared) Direct dependencies (what's explicitly in package.json) Transitive dependencies (what your dependencies need) C# // Read package.json dependencies var directRanges = ReadDirectDependencyRanges(packageJsonContent); // For each direct dep, look it up in the lock file foreach (var (name, range) in directRanges) { var pkg = Resolve(name, range, parsedLock); if (pkg != null) { // It's resolved to version X.Y.Z direct.Add(new ResolvedDependency(pkg.Name, pkg.Version, range)); // Queue it to traverse its dependencies queue.Enqueue(pkg); } } // Depth-first traversal to collect transitives while (queue.TryDequeue(out var pkg)) { foreach (var (depName, depRange) in pkg.DependencyRanges) { var dep = Resolve(depName, depRange, parsedLock); if (dep != null && !visited.Contains($"{dep.Name}@{dep.Version}")) { transitive.Add(...); queue.Enqueue(dep); } } } Result: Three lists of ResolvedDependency objects with exact versions and requested ranges. Silverfish uses this to build the full dependency graph in its UI. Step 5: Handle Monorepos Monorepos have multiple package.json files. The key insight: walk up the directory tree to find the root lock file. C# static IEnumerable AncestorDirs(string dir) { var current = dir; while (true) { yield return current; if (string.IsNullOrEmpty(current)) break; current = Path.GetDirectoryName(current); } } So packages/web/package.json in an entria-style monorepo correctly finds the root yarn.lock instead of failing. Each workspace member gets its own component record in Silverfish. How the Silverfish IDP Uses This Once the analyzer extracts all this metadata, it: Maps dependencies visually — showing which components depend on what Flags version mismatches — when different packages pin different versions of the same library Detects tech stacks — knowing which services are frontend, which are backend, which databases they use Tracks upgrades — identifying outdated packages and planning coordinated updates Enables governance — enforcing policies like "no direct jquery dependencies" or "all frontends must use React 18+" Lessons Learned Abstraction beats assumptions: I wrote the whole thing to accept Func> readFileContentAsync instead of directly reading files. This made it testable and backend-agnostic (GitHub API, filesystem, cache, whatever). Format-specific parsing is worth it: I could have given up on Yarn/pnpm/Bun and only parsed npm lock files. But each format's parser is ~100-150 lines and handles real repos that exist in the wild. Conflicts are data, not errors: Instead of failing when I find multiple lock files, I report them. That's valuable information ("why do you have both yarn.lock and package-lock.json?"). Monorepos are normal: Walking ancestor directories
for lock files + detecting internal workspace packages turned out to be essential, not an edge case. Version constraints matter: Storing both the requested range (^1.2.3) and resolved version (1.2.5) proved useful—you can detect upgradeable deps without breaking changes. What's Next The JS/TS analyzer is one piece of Silverfish's language support. It already has support for .NET languages and Ruby. I'll be building similar analyzers for Python, Go, Java, and other ecosystems. The pattern is the same: detect the package manager, identify components, resolve dependencies, extract versions. If you're trying to understand complex multi-language codebases at scale, this approach should help. The code is C# 14 with only standard library dependencies—no bloat. <!-- SC_ON --> submitted by /u/DavidArno (https://www.reddit.com/user/DavidArno)
[link] (https://dashboard.silverfishsoftware.com/documentation) [comments] (https://www.reddit.com/r/programming/comments/1suhkea/how_i_built_an_automated_jsts_repository_analyzer/)
Modern LZ Compression Part 2: FSE and Arithmetic Coding
https://www.reddit.com/r/programming/comments/1suioko/modern_lz_compression_part_2_fse_and_arithmetic/

<!-- SC_OFF -->This is the second article in a series discussing modern compression techniques. The first one covered Huffman + LZ. This one covers optimal entropy coders (FSE and Arithmetic), and some additional tricks to get closer to the state of the art. The full compressor and decompressor are just over 1500 lines of pretty compact C++: https://github.com/glinscott/linzip2/blob/master/main.cc. It's been seven years since the first article! Hopefully not so long before the third (and probably final one). Part 1 discussion thread: https://www.reddit.com/r/programming/comments/amfzqg/modern_lz_compression/ <!-- SC_ON --> submitted by /u/glinscott (https://www.reddit.com/user/glinscott)
[link] (https://glinscott.github.io/lz/part2.html) [comments] (https://www.reddit.com/r/programming/comments/1suioko/modern_lz_compression_part_2_fse_and_arithmetic/)
Caching Beyond Redis: Real-World Strategies That Don’t Break Your System
https://www.reddit.com/r/programming/comments/1sv664k/caching_beyond_redis_realworld_strategies_that/

<!-- SC_OFF -->In the article, I break down:
• why caching is really a trade-off between speed and correctness
• when to use in-memory cache, Redis-style distributed cache, and CDN caching
• cache-aside, write-through, write-back, and read-through with real examples
• cache invalidation, stale data, and cache stampedes
• when caching is the wrong solution entirely <!-- SC_ON --> submitted by /u/anant94 (https://www.reddit.com/user/anant94)
[link] (https://commitlog.cc/posts/caching-beyond-redis) [comments] (https://www.reddit.com/r/programming/comments/1sv664k/caching_beyond_redis_realworld_strategies_that/)