Galactic Grid

By Chris, Lead Product & UX Architect (with Scott, Lead Data Engineer)

When developers set out to build a data-driven trivia engine or a daily grid puzzle around a famous fictional universe, their first hurdle is almost always the data pipeline. You need thousands of characters, droids, starships, planets, and factions structured into relational arrays. You need to know exactly which entities appeared in which specific films or television series.

Where do you get that data?

In the early planning phases of our app, before a single React component was created or a single CSS stylesheet was written, solving the data problem was our absolute top priority. In fact, writing our initial scraping and database ingestion pipeline alongside Scott was literally the very first code we ever executed on the project.

Some developers might naturally look toward official corporate API endpoints or attempt to scrape proprietary studio websites. To us, that path was an immediate non-starter. Scraping official corporate servers to populate a custom web application is a legal minefield. It exposes your project to claims of unauthorized server access, database rights infringement, and direct commercial conflict.

Instead, we turned to the ultimate monument of fan scholarship: Wookieepedia.

The Power of Community-Sourced Lore

Wookieepedia is a sprawling, volunteer-maintained encyclopedia built by thousands of passionate fans over more than two decades. Every single article, category tag, and appearance log on the platform is written, edited, and verified by community members who love the lore.

The text content on Wookieepedia is hosted under the Creative Commons Attribution-ShareAlike License (CC BY-SA 3.0). This open-licensing framework explicitly permits the public to share, adapt, and build upon the text, provided that proper attribution is given and derivative works respect the open ecosystem.

Choosing a fan-curated wiki over an official corporate database gave us three immediate advantages:

Ethical Data Provenance: We were referencing open, public fan scholarship rather than attempting to reverse-engineer proprietary studio databases.

Comprehensive Coverage: Hardcore fans document background details, obscure dialogue references, and minor character appearances far more thoroughly than any official corporate marketing site ever would.

A Self-Healing Data Ecosystem: Because Wookieepedia is a living, community-editable platform, any factual gaps or missing appearance records can be updated openly by fans, players, or developers at any time.

// Example MongoDB Lore Document Structured From Fan Wiki Ingestion

interface ScrapedWikiEntity {

\_id: string;

canonicalTitle: string; // e.g., "Mon Mothma"

primaryCategory: string; // e.g., "character"

appearanceTitles: string\[]; // e.g., \["Rogue One", "Andor", "Return of the Jedi"]

wookieepediaUrl: string; // Direct link to the open CC-BY-SA wiki article

lastScrapedTimestamp: Date;

}

Handling Taxonomy Clutter with an Objective Validation Rule

If you have ever explored a public wiki, you know that community taxonomy can quickly become messy. A single search query might bring up major primary characters, minor background extras, fictional sub-species, military ranks, or obscure lore concepts all lumped together under general tags.

When building a daily 3x3 matrix game where every cell query must evaluate to a strict TRUE or FALSE boolean result, database clutter can create player frustration. If a player types in a character they distinctly remember seeing on screen, but the game marks it wrong because our database missed the relationship, the trust in our gameplay loop breaks.

Scott and I needed a simple, completely objective validation rule to govern what constitutes a correct answer for any given grid cell.

We established what we call The Wookieepedia Appearance Standard:

An entity is recognized as a valid answer for a given media title if and only if it has its own dedicated article on Wookieepedia, AND that specific media title is explicitly listed in the "Appearances" section on its article page.

This rule eliminated all subjectivity. We did not have to debate among ourselves whether a character's brief non-speaking background cameo counted as an appearance. We delegated the truth standard entirely to the published consensus of the fan community archive.

// Core Answer Validation Logic inside our API Route

export function validateGridAnswer(

entity: ScrapedWikiEntity,

targetMediaTitle: string,

targetCategory: string

): boolean {

// Check if the entity belongs to the requested category (e.g., 'character', 'droid')

const matchesCategory = entity.primaryCategory === targetCategory;

// Check if the target movie or show exists in the entity's Wookieepedia Appearances list

const matchesAppearance = entity.appearanceTitles.some(

(title) => title.toLowerCase() === targetMediaTitle.toLowerCase()

);

return matchesCategory && matchesAppearance;

}

The Crowd-Sourced Resolution Loop

What happens when a player attempts an answer that they know is factually correct from watching a show, but the game marks it incorrect because Wookieepedia's editors had not yet added that title to the character's Appearances section?

In a traditional closed video game, that bug would sit in a backlog for months waiting for a studio developer to release a patch. In our open data architecture, it creates a fascinating community feedback loop.

If a player encounters an answer that fails validation in Galactic Grid, they can take two actions:

Notify Our Team: They can drop us a note through our contact channels, and Scott or I can inspect the entry and update our database pipeline.

Update Wookieepedia Directly: Because Wookieepedia is a free-to-edit public wiki, the player can jump directly over to the character's page, add the missing media title to the Appearances section with a proper source citation, and submit the edit to the community.

Once the community approves the edit on Wookieepedia, our scraper ingests the updated page data. The next time that specific cell combination appears on a daily board, the player's answer will validate perfectly.

This structure transforms our players from passive consumers into active contributors to public lore preservation. It turns potential gameplay arguments into collaborative scholarship.

Data Provenance as a Legal Shield

In intellectual property law, the concept of Data Provenance refers to the documented origin, chain of custody, and historical sourcing of a dataset. If a studio, rights holder, or legal representative ever inquires about how an application indexes fictional lore, having a clear, documented chain of data provenance is your strongest shield.

If we had scraped official corporate applications or proprietary API endpoints, our data chain would be tainted by unauthorized extraction. But by sourcing our relational matrix exclusively from Wookieepedia, our legal standing is reinforced across multiple statutory pillars:

Separation from Proprietary Datastores

We do not use, copy, or touch official corporate databases. Our application operates entirely on publicly accessible, fan-documented factual reference points. Facts, historical dates, and relational lists are not copyrightable under United States law (Feist Publications, Inc. v. Rural Telephone Service Co.).

2. Alignment with Creative Commons

By referencing text entries released under Creative Commons Attribution-ShareAlike (CC BY-SA 3.0), our data sourcing respects open-source web standards. We cite our sources transparently, display attribution headers on our home screen, and link every post-game debrief card directly back to the original Wookieepedia article pages.

3. Strengthening Transformative Educational Fair Use

Under the four-factor test of Fair Use (17 U.S.C. § 107), Factor 1 evaluates the purpose and character of the use, while Factor 4 evaluates the effect upon the potential market for the original work.

By indexing fan-curated wiki data and embedding direct links to Wookieepedia articles across our post-game results, our application functions as an educational discovery engine. We do not substitute for watching the movies or reading the books. Instead, we drive traffic directly to public research archives and fan encyclopedias, encouraging players to read about the lore, study background histories, and support open community documentation.

Key Takeaways for Indie Developers

If you are an indie creator or vibe coder planning to build a trivia, search, or research application around a sprawling pop-culture universe, your choice of data pipeline will define your project's longevity.

Avoid Proprietary Scraping: Never attempt to scrape official corporate endpoints or proprietary databases. The short-term convenience is never worth the severe legal liability.

Embrace Community Wikis: Source factual metadata from open, community-curated fan wikis like Wookieepedia. Their depth is unmatched, and their Creative Commons frameworks provide a clear path for respectful reuse.

Establish Objective Validation Rules: Define clear, automated criteria for answer correctness based on published wiki page sections, such as dedicated article pages and explicit appearance lists.

Leverage Data Provenance as Protection: Document your data pipeline openly. Display attribution headers, link back to source articles, and highlight the educational, scholarship-focused nature of your platform.

By grounding our application in open fan data, Scott and I built a database that is morally sound, legally protected, and powered by the passion of thousands of fans worldwide.

Public Wiki Scraping vs. Official Databases: The Data Provenance Strategy