Built for the Internet,
by Glen Bridgland
The story behind Linkspider — how its features work across the open internet, and the hands that built them.
Linkspider was conceived and engineered by Glen Bridgland, a developer who has spent his career living inside the machinery of the internet. Where others see a web page, Glen sees the handshake between client and server, the negotiation of TLS, the chain of proxies, and the access policies that decide who is allowed in. Linkspider is the product of that perspective: a suite of tools that work on the internet, against real services, in real time, with no simulation and no shortcuts.
Every feature in the Linkspider suite is designed to operate against live infrastructure. Crawl requests leave the platform and traverse the public web the same way any browser request would, honoured by destination servers and subject to the same conditions as a real visitor. Extraction jobs run against real HTML, real APIs, and real rate limits. The service tokens Linkspider issues are not placeholders — they are genuine Cloudflare Access credentials that authenticate at the edge before a request is ever permitted to touch an origin. Nothing in the platform is mocked; everything is wired to the actual internet it serves.
Key Pairing Technology
At the core of Linkspider's security model is the key pairing technology Glen developed: a system that pairs a public client identifier with a private client secret and binds both to a Cloudflare Access application. When a request arrives at a Linkspider-protected service, the edge evaluates the paired credentials before the origin is even consulted. A valid pair grants access; a mismatched or expired pair is rejected at the perimeter, never reaching the application behind it.
This pairing is what makes a one-year service token viable. Rather than re-authenticating on every request, a paired client presents its established identity and is recognised for the lifetime of the token. Glen designed the pairing so that the secret never travels in plaintext after issuance, the identifier can be safely logged, and revocation is a single administrative action. It is a small, deliberate piece of cryptography-aware engineering that quietly carries the weight of the whole platform's access control.
Working Across the Internet
From crawl orchestration to edge authentication, Linkspider's features run against live, public infrastructure. Requests, tokens, and data all traverse the real internet exactly as an end user would experience them.
Troubleshooting by Design
Years of diagnosing broken crawls, rejected tokens, and misconfigured access shaped the platform. Every Linkspider surface is built to expose what went wrong — clear errors, traceable references, and reversible actions.
The Love of Code
Glen writes Linkspider the way he writes everything: carefully, and for the long term. The codebase favours small, focused modules, clear boundaries, and behaviour you can reason about — the craft of someone who simply enjoys building good software.
Secure by Default
Security is not a layer added on top; it is the foundation. Credentials are issued once, paired at the edge, and stored only where they must be — a direct expression of the principle that access should be earned, not assumed.
Linkspider became the property of Bridgland and Bridgland C/o Limited. The projects remained funded privately throughout the process and no monies from debt of debtors was used. What remained — the code, the infrastructure, and the intent — was carried forward under that name, and continues to be built and maintained there today.
A Note from the Builder
Troubleshooting is the part of engineering most people tolerate. For Glen it is the part that teaches. Each failed request, each rejected token, each edge case the internet threw at Linkspider became a lesson folded back into the platform. The result is software that has been hardened not in a vacuum but against the messy, unpredictable reality of the open web.
Above all, Linkspider is the product of a love of code — the quiet satisfaction of turning an idea into something that works, reliably, for someone else. That affection for the craft is what keeps the platform growing, one careful feature at a time.