Skip to content

Add query endpoint for trusted users#97

Open
MialLewis wants to merge 7 commits intomainfrom
add_query_endpoint
Open

Add query endpoint for trusted users#97
MialLewis wants to merge 7 commits intomainfrom
add_query_endpoint

Conversation

@MialLewis
Copy link
Contributor

@MialLewis MialLewis commented Jan 29, 2026

This PR adds an endpoint that allows trusted users to query the report database.

  • It requires a token which will only be shared selectively
  • It utilises a read only Postgres account
  • The database connection is read only

To test (contact me for the token, it's on keeper):

Error on write:

curl -vk "https://reports.a.staging-mantidproject.stfc.ac.uk/api/query" \
  --data-urlencode "sql=INSERT INTO services_featureusage DEFAULT VALUES" --data-urlencode "token=<TOKEN>"

Read action:

curl -vk "https://reports.a.staging-mantidproject.stfc.ac.uk/api/query" \
  --data-urlencode "sql=SELECT * FROM services_featureusage LIMIT 10;" --data-urlencode "token=<TOKEN>" 

Closes #95

@MialLewis MialLewis changed the base branch from main to make_changes_for_django_5110 January 29, 2026 17:00
Base automatically changed from make_changes_for_django_5110 to main January 30, 2026 13:34
Copy link
Contributor

@warunawickramasingha warunawickramasingha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proposed a few suggestions to improve the code, style and better handle exceptions + rate limiting rule at nginx level. Please check and apply as necessary.

def verify_token(request) -> bool:
token = get_bearer_token(request)
secret = environ.get("QUERY_SECRET_KEY", "")
return token == secret
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a huge concern in this context. But for secret validations, it is recommended to use something like compare_digest instead of == to avoid timing attacks.

Suggested change
return token == secret
from hmac import compare_digest
return compare_digest(token, secret)

# Create your views here.
from django.views.decorators.cache import cache_page
from services.models import Message, Usage, FeatureUsage, Location
from rest_framework import response, viewsets
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from rest_framework import response, viewsets, status

Comment on lines +50 to +52
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess these numbers are quite generous

Suggested change
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_connect_timeout 30s;
proxy_send_timeout 30s;
proxy_read_timeout 30s;

@api_view(("POST",))
def query(request, format=None):
if not verify_token(request):
return response.Response(status=401, data="UNAUTHORIZED")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For improved readability

Suggested change
return response.Response(status=401, data="UNAUTHORIZED")
return response.Response(status=status.HTTP_401_UNAUTHORIZED, data="UNAUTHORIZED")

Comment on lines +339 to +345
if sql_err:
return response.Response(status=400, data=f"Invalid Parameters: {sql_err}")
conn = connections["readonly"]
with conn.cursor() as cur:
cur.execute(sql)
res = cur.fetchall()
return response.Response(res)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception handling for db operations

Suggested change
if sql_err:
return response.Response(status=400, data=f"Invalid Parameters: {sql_err}")
conn = connections["readonly"]
with conn.cursor() as cur:
cur.execute(sql)
res = cur.fetchall()
return response.Response(res)
if sql_err:
return response.Response(status=status.HTTP_400_BAD_REQUEST, data=f"Invalid Parameters: {sql_err}")
try:
conn = connections["readonly"]
with conn.cursor() as cur:
cur.execute(sql)
res = cur.fetchall()
except Exception:
return Response({"error": "Query failed"}, status=status.HTTP_400_BAD_REQUEST)
return response.Response(res)

err = ""
if not val:
err = f"No {param} parameter provided"
return err, val
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To work better with --data-urlencode "sql="

Suggested change
return err, val
def get_parameter(request, param):
val = request.POST.get(param)
if val is None or val.strip() == "":
return f"No {param} parameter provided", None
return None, val

@@ -1 +1 @@
server {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Define a rate limiting zone with 5 requests per second to be used at /api/query location to avoid any spikes due to usage errors

Suggested change
http {
limit_req_zone $binary_remote_addr zone=query_limit:10m rate=5r/s;
server {

# allow ISIS VPN traffic
allow 130.246.0.0/16;
deny all;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apply the above defined query_limit rate limiting allowing short bursts with delays to be queued further

Suggested change
# Rate limiting
limit_req zone=query_limit burst=10;

proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add extra closing } for the new http block

Suggested change
}
}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants