Briefs with Bigdata.com

This repository contains a docker image for running a brief generating service using Bigdata.com SDK. You can read more on our docs.

What are Briefs in Bigdata?

Briefs are concise, automated reports generated by Bigdata's platform that help users stay informed about their watchlists and areas of interest. They are personalized market updates that deliver only the most relevant insights based on your watchlist, helping you stay informed without information overload. Unlike traditional newsfeeds or generic newsletters, they're tailored to your interests and can be delivered when you preferred, whether that's daily, weekly or any other frequency you choose.

Prerequisites

• A Bigdata.com account that supports programmatic access.
• A Bigdata.com API key, which can be obtained from your account settings.
  • For more information on how to get an API key, refer to the Bigdata.com documentation.
• Companies to generate briefs for, specified either as:
  • A watchlist ID (UUID) with the list of companies you want to generate briefs for.
    • A watchlist can be created using the Bigdata.com SDK or through the app @ https://app.bigdata.com/watchlists.
  • A list of entity IDs (e.g., ["D8442A"]) - RP_ENTITY_ID format

Quickstart

To quickly get started, you have two options:

1. Build and run locally: You need to build the docker image first and then run it:

# Clone the repository and navigate to the folder
git clone git@github.com:Bigdata-com/bigdata-briefs.git
cd "bigdata-briefs"

# Build the docker image
docker build -t bigdata_briefs .

# Run the docker image
docker run -d \
  --name bigdata_briefs \
  -p 8000:8000 \
  -e BIGDATA_API_KEY=<bigdata-api-key-here> \
  -e OPENAI_API_KEY=<openai-api-key-here> \
  bigdata_briefs

2. Run directly from GitHub Container Registry:

docker run -d \
  --name bigdata_briefs \
  -p 8000:8000 \
  -e BIGDATA_API_KEY=<bigdata-api-key-here> \
  -e OPENAI_API_KEY=<openai-api-key-here> \
  ghcr.io/bigdata-com/bigdata-briefs:latest

This will start the brief service locally on port 8000. You can then access the service @ http://localhost:8000/ and the documentation for the API @ http://localhost:8000/docs.

For a custom enterprise-ready solution, please contact us at support@bigdata.com

Security Measures

We perform a pre-release security scan on our container images to detect vulnerabilities in all components.

How to use? Generate a brief for your Bigdata.com watchlist

A brief is an executive summary of financially relevant information about a set of companies that form your watchlist.

Using the UI

There is a very simple UI available @ http://localhost:8000/.

Set your watchlist ID, the relevant dates for your report and whether you want to filter the brief only to novel information based on previously generated briefs and click on the "Generate Brief" button.

Programmatically

You can generate a brief by sending a POST request to the /briefs/create endpoint with the required parameters.

Basic Example - Using a Watchlist ID

curl -X 'POST' \
  'http://127.0.0.1:8000/briefs/create' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "entities": "db8478c9-34db-4975-8e44-b1ff764098ac",
  "report_start_date": "2025-09-23T15:00:00",
  "report_end_date": "2025-09-30T15:00:00"
}'

Basic Example - Using Entity IDs

curl -X 'POST' \
  'http://127.0.0.1:8000/briefs/create' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "entities": ["D8442A"],
  "report_start_date": "2025-10-27",
  "report_end_date": "2025-11-03"
}'

Advanced Example - With Custom Topics and Configuration

curl -X 'POST' \
  'http://127.0.0.1:8000/briefs/create' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "entities": ["D8442A", "168A5D"],
  "report_start_date": "2025-10-27",
  "report_end_date": "2025-11-03",
  "novelty": true,
  "sources": null,
  "categories": null,
  "topics": [
    "What notable changes in {entity}\u0027s financial performance metrics have been reported recently?",
    "Has {entity} revised its financial or operational guidance for upcoming periods?",
    "What significant strategic initiatives or business pivots has {entity} announced recently?"
  ],
  "source_rank_boost": 10,
  "freshness_boost": 8,
  "disable_introduction": true
}'

Key Parameters:

• entities: Either a watchlist ID (UUID string) or a list of entity IDs (e.g., ["D8442A"])
• report_start_date / report_end_date: Date range for the briefing (ISO format: YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS)
• novelty (optional): Filter for only new or unique information based on previously generated briefs
• topics (optional): List of research questions/topics. Use {entity} placeholder to customize per entity
• sources (optional): List of source IDs to whitelist, all other sources will be filtered out (default: all sources)
• categories (optional): List of category IDs to whitelist, all other categories will be filtered out (default: all categories). Possible categories are: expert_interviews, filings, news, podcasts, research, transcripts and my_files.
• source_rank_boost (optional): Control how sources are prioritized (default: 10)
• freshness_boost (optional): Control freshness weighting for sources (default: 8)
• disable_introduction (optional): Skip the introduction section in the brief

Warning

The novelty feature operates using knowledge derived from previously generated briefs for each entity. Consequently, a new deployment with an empty database will have no prior context, and the novelty filter will not exclude any information until briefs have been created and stored. To ensure novelty content, make sure your database persists across service restarts. If necessary, consider pre-populating it by generating initial briefs for your entities covering previous periods.

The response will include a request_id. You can check the status and get your brief with:

curl -X GET \
  'http://localhost:8000/briefs/status/<request_id>' \
  -H 'accept: application/json'

For more details on all available parameters, refer to the API documentation @ http://localhost:8000/docs.

Large-Scale Portfolio Briefs Generation

For generating briefs for large portfolios (hundreds or thousands of companies), see the Large-Scale Portfolio Briefs Generation notebook in the bigdata-cookbook repository.

This notebook demonstrates how to:

• Process large numbers of entities in configurable batches
• Load entity identifiers from CSV files
• Customize topics and research questions for different batches
• Monitor batch processing with status polling
• Export results to JSON and Excel formats

The notebook uses this briefs service API to generate briefs programmatically, making it ideal for portfolio managers and analysts who need to monitor many entities simultaneously. The service can handle large numbers of entities in a single request, and the notebook shows how to organize batch processing for scheduling across time zones or running concurrent service instances.

Enable access token security

You can optionally protect the API endpoints using an access token. To enable this feature, set the ACCESS_TOKEN environment variable when running the Docker container. For example:

docker run -d \
  --name bigdata_briefs \
  -p 8000:8000 \
  -e BIGDATA_API_KEY=<bigdata-api-key-here> \
  -e OPENAI_API_KEY=<openai-apikey-here> \
  -e ACCESS_TOKEN=<access-token-here> \
  ghcr.io/bigdata-com/bigdata_briefs:latest

Then all API requests must include a token query parameter with the correct value to be authorized. For example:

curl -X POST \
  'http://localhost:8000/briefs/create?token=<access-token-here>' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "entities": ["D8442A"],
  "report_start_date": "2025-10-27",
  "report_end_date": "2025-11-03"
}'

Install and for development locally

uv sync --dev

To run the service, you need an API key from Bigdata.com set on the environment variable BIGDATA_API_KEY and additionally provide an API key from a supported LLM provider, for now OpenAI.

# Set environment variables
export BIGDATA_API_KEY=<bigdata-api-key-here>
export OPENAI_API_KEY=<openai-api-key-here>

Then, the following command will start the brief service locally on port 8000.

uv run -m bigdata_briefs

Tooling

This project uses ruff for linting and formatting and ty for a type checker. To ensure your code adheres to the project's style guidelines, run the following commands before committing your changes:

make type-check
make lint
make format