refactor: Replace Arduino HTTP Client with esp native #391

hhvrc · 2025-12-04T02:27:58Z

No description provided.

changeset-bot · 2025-12-04T02:28:02Z

⚠️ No Changeset found

Latest commit: 6b85a36

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

src/http/HTTPClientState.cpp

src/util/ParitionUtils.cpp

Copilot

Pull request overview

This pull request replaces the Arduino HTTP Client library with the ESP-IDF native HTTP client implementation. The change represents a significant architectural shift towards using ESP32's built-in networking capabilities, improving platform integration and potentially reducing dependency on third-party libraries.

Key Changes:

Introduced new HTTP client architecture with HTTPClient, HTTPClientState, HTTPResponse, and JsonResponse classes
Replaced string_view URL parameters with const char* for HTTP client APIs
Refactored rate limiting into a separate RateLimiters module
Removed the old HTTPRequestManager implementation and replaced all usages throughout the codebase

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
src/util/DomainUtils.cpp	New utility to extract domain from URL (critical bug: missing assignment on line 11)
src/util/ParitionUtils.cpp	Updated to use new HTTP client; logic error in status code check (line 63)
src/serialization/JsonAPI.cpp	Removed unused `code` parameter from JSON parsing functions
src/serial/command_handlers/domain.cpp	Updated to use new HTTP client with error handling
src/serial/command_handlers/authtoken.cpp	Updated to use new HTTP client in scoped block
src/serial/SerialInputHandler.cpp	Removed unused import
src/http/RateLimiters.cpp	New rate limiter management module extracted from old implementation
src/http/JsonAPI.cpp	Refactored to use new HTTP client, simplified header setting
src/http/HTTPRequestManager.cpp	Deleted - replaced by new implementation
src/http/HTTPClientState.cpp	New ESP-IDF HTTP client state management (bug in constructor initialization line 21, log message bug line 181)
src/http/HTTPClient.cpp	New HTTP client facade providing Get and GetJson methods
src/OtaUpdateManager.cpp	Updated all HTTP requests to use new client
src/GatewayConnectionManager.cpp	Updated API calls to use new HTTP client with improved error handling
include/util/PartitionUtils.h	Changed parameter type from string_view to const char*
include/util/DomainUtils.h	New header for domain extraction utility
include/serialization/JsonAPI.h	Updated parsing function signatures to remove `code` parameter, forward declared cJSON
include/http/*.h	New HTTP client interface headers (HTTPError.h has critical bug: empty error strings)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/util/DomainUtils.cpp

src/util/ParitionUtils.cpp

include/http/HTTPError.h

src/util/DomainUtils.cpp

include/http/HTTPClient.h

src/http/HTTPClientState.cpp

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

hhvrc · 2025-12-09T11:48:25Z

Cannot merge until we switch over to ESP-IDF due to inability configure global certs when arduino es-idf framework has precompiled binaries

Copilot

Pull request overview

Copilot reviewed 27 out of 27 changed files in this pull request and generated 11 comments.

Comments suppressed due to low confidence (1)

platformio.ini:48

These configuration flags completely disable SSL/TLS certificate verification, making all HTTPS connections vulnerable to man-in-the-middle attacks. This is a serious security vulnerability that should not be enabled in production code.

Consider using proper certificate validation with either a CA bundle or certificate pinning instead of disabling verification entirely.

  -DCONFIG_ESP_TLS_INSECURE=y
  -DCONFIG_ESP_TLS_SKIP_SERVER_CERT_VERIFY=y

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-15T15:29:55Z

include/http/HTTPClient.h

+    HTTPClient(const char* url, uint32_t timeoutMs = 10'000)
+      : m_state(std::make_shared<HTTPClientState>(url, timeoutMs))
+    {
+    }


The new HTTPClient implementation does not integrate with the rate limiting system. The old implementation called _getRateLimiter() and checked rateLimiter->tryRequest() before making requests. Without this integration, the application may exceed API rate limits and get blocked by the server. Consider integrating the RateLimiters::GetRateLimiter functionality into the HTTPClient.

Copilot · 2025-12-15T15:29:55Z

src/OtaUpdateManager.cpp

+
+  auto content = response.ReadString();
+  if (content.error != HTTP::HTTPError::None) {
+    OS_LOGE(TAG, "Failed to fetch hashes: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());


Same issue - attempting to log content.data.c_str() when an error has occurred. The content data may not be valid when content.error is not None.

Suggested change

OS_LOGE(TAG, "Failed to fetch hashes: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());

OS_LOGE(TAG, "Failed to fetch hashes: %s [%u]", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

Copilot · 2025-12-15T15:29:55Z

src/GatewayConnectionManager.cpp

-    OS_LOGE(TAG, "Unexpected response code: %d", response.code);
+  auto content = response.ReadJson();
+  if (content.error != HTTP::HTTPError::None) {
+    OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());


Same issue - using response.Error() when the error occurred during content reading. Should use content.error instead.

Suggested change

OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(content.error), response.StatusCode());

Copilot · 2025-12-15T15:29:56Z

src/GatewayConnectionManager.cpp

-    OS_LOGE(TAG, "Unexpected response code: %d", response.code);
+  auto content = response.ReadJson();
+  if (content.error != HTTP::HTTPError::None) {
+    OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());


Same issue - using response.Error() when the error occurred during content reading. Should use content.error instead.

Suggested change

OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(content.error), response.StatusCode());

Copilot · 2025-12-15T15:29:56Z

src/http/HTTPClientState.cpp

+
+  if (key == "retry-after") {
+    uint32_t seconds = 0;
+    if (!Convert::ToUint32(value, seconds) || seconds <= 0) {


The condition seconds <= 0 will always be false because seconds is an unsigned integer (uint32_t). If conversion fails and returns 0, this check won't catch it. Consider checking if the conversion succeeded first, or use a different approach to validate the retry-after value.

Suggested change

if (!Convert::ToUint32(value, seconds) || seconds <= 0) {

if (!Convert::ToUint32(value, seconds) || seconds == 0) {

Copilot · 2025-12-15T15:29:57Z

src/serial/command_handlers/domain.cpp

-  OS_LOGI(TAG, "Successfully connected to \"%.*s\", version: %s, commit: %s, current time: %s", arg.length(), arg.data(), resp.data.version.c_str(), resp.data.commit.c_str(), resp.data.currentTime.c_str());
+  auto content = response.ReadJson();
+  if (content.error != OpenShock::HTTP::HTTPError::None) {
+    SERPR_ERROR("Tried to read response from backend, but failed (%s), refusing to save domain to config", OpenShock::HTTP::HTTPErrorToString(response.Error()));


The error message calls response.Error() but should call content.error since the error occurred during the ReadJson operation, not during the initial request. The response's Error() method may not reflect the actual error that occurred during content reading.

Suggested change

SERPR_ERROR("Tried to read response from backend, but failed (%s), refusing to save domain to config", OpenShock::HTTP::HTTPErrorToString(response.Error()));

SERPR_ERROR("Tried to read response from backend, but failed (%s), refusing to save domain to config", OpenShock::HTTP::HTTPErrorToString(content.error));

Copilot · 2025-12-15T15:29:57Z

src/OtaUpdateManager.cpp

+
+  auto content = response.ReadString();
+  if (content.error != HTTP::HTTPError::None) {
+    OS_LOGE(TAG, "Failed to fetch list: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());


The error logging is inconsistent - it uses response.Error() and response.StatusCode() but also tries to log content.data.c_str(). However, if content.error is not None, then content.data may be empty or invalid. Consider logging only the error information without attempting to log the potentially invalid content data.

Suggested change

OS_LOGE(TAG, "Failed to fetch list: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());

OS_LOGE(TAG, "Failed to fetch list: %s [%u]", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

Copilot · 2025-12-15T15:29:58Z

src/OtaUpdateManager.cpp

+
+  auto content = response.ReadString();
+  if (content.error != HTTP::HTTPError::None) {
+    OS_LOGE(TAG, "Failed to fetch firmware version: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());


Same issue as line 444 - attempting to log content.data.c_str() when content.error indicates an error occurred. The content data may be empty or invalid in this case.

Suggested change

OS_LOGE(TAG, "Failed to fetch firmware version: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());

OS_LOGE(TAG, "Failed to fetch firmware version: %s [%u]", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

Copilot · 2025-12-15T15:29:58Z

src/GatewayConnectionManager.cpp

-    OS_LOGE(TAG, "Unexpected response code: %d", response.code);
+  auto content = response.ReadJson();
+  if (content.error != HTTP::HTTPError::None) {
+    OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());


The error message calls response.Error() when the error actually occurred during ReadJson(), so it should reference content.error instead. The response's Error() method will likely return HTTPError::None since the initial request succeeded.

Suggested change

OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(content.error), response.StatusCode());

platformio.ini

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

github-actions · 2025-12-17T13:04:29Z

Cpp-Linter Report ⚠️

Some files did not pass the configured checks!

clang-format (v18.1.3) reports: 12 file(s) not formatted

include/http/HTTPClient.h
include/http/HTTPClientState.h
include/http/HTTPError.h
include/http/HTTPResponse.h
include/http/JsonResponse.h
include/http/RateLimiters.h
include/http/ReadResult.h
src/GatewayConnectionManager.cpp
src/http/HTTPClientState.cpp
src/main.cpp
src/util/DomainUtils.cpp
src/util/ParitionUtils.cpp

Have any feedback or feature suggestions? Share it here.

hhvrc added 10 commits November 14, 2024 15:08

Initial commit

657ab37

Merge branch 'develop' into feature/ditch-arduino-httpclient

ff81cf6

Merge branch 'develop' into feature/ditch-arduino-httpclient

22dc664

Use helpers

8ceac76

Merge branch 'develop' into feature/ditch-arduino-httpclient

042f28d

Merge branch 'develop' into feature/ditch-arduino-httpclient

50e317f

Some more stuff

9dcc162

Merge branch 'develop' into feature/ditch-arduino-httpclient

49be717

Push WIP

eb734c7

More work

c342d31

hhvrc added this to the ESP-IDF milestone Dec 4, 2025

hhvrc self-assigned this Dec 4, 2025

hhvrc added the type: enhancement label Dec 4, 2025

github-project-automation bot added this to Roadmap Dec 4, 2025

github-project-automation bot moved this to Todo in Roadmap Dec 4, 2025

hhvrc and others added 14 commits December 4, 2025 03:28

Merge branch 'develop' into feature/ditch-arduino-httpclient

664a589

More work

9604c39

More work on custom http client

d7774cc

More wooooorkkkkk AAAAAAAAAAAAAAAAAAA

0d966e3

These are not needed lmao

979a17d

nvm...

8cc417b

more work

c0ee503

More fixes and touchup

e8d2790

Update HTTPClientState.cpp

d83d888

More work

3a274f5

Arduino is and always has been a shitty ecosystem

bb36316

Merge branch 'develop' into feature/ditch-arduino-httpclient

c44bbc0

Revert nitpick

0bfc377

Revert more nitpicking

454a508

hhvrc marked this pull request as ready for review December 5, 2025 11:30

chatgpt-codex-connector bot reviewed Dec 5, 2025

View reviewed changes

src/http/HTTPClientState.cpp Show resolved Hide resolved

src/util/ParitionUtils.cpp Show resolved Hide resolved

LucHeart requested a review from Copilot December 5, 2025 11:51

Copilot started reviewing on behalf of LucHeart December 5, 2025 11:52 View session

Copilot finished reviewing on behalf of LucHeart December 5, 2025 11:54

Copilot AI reviewed Dec 5, 2025

View reviewed changes

hhvrc and others added 4 commits December 5, 2025 14:07

Pass along headers and Retry-After

a6bc157

Fix more issues

7e79d41

Last touchup

614e4c8

Update include/http/HTTPClient.h

68cf530

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

hhvrc requested review from LucHeart and nullstalgia December 5, 2025 13:46

hhvrc and others added 5 commits December 5, 2025 14:49

Update src/util/DomainUtils.cpp

b59e4b9

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/util/DomainUtils.cpp

435e6ef

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/serial/command_handlers/authtoken.cpp

3579cee

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/util/ParitionUtils.cpp

c9f2094

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Fixed url assignment issues

da86842

hhvrc added the status: blocked An issue that is blocked by another issue or dependency label Dec 9, 2025

hhvrc changed the title ~~feat: Replace Arduino HTTP Client with esp native~~ refactor: Replace Arduino HTTP Client with esp native Dec 9, 2025

LucHeart requested a review from Copilot December 15, 2025 14:57

Copilot started reviewing on behalf of LucHeart December 15, 2025 15:24 View session

Copilot AI reviewed Dec 15, 2025

View reviewed changes

hhvrc and others added 4 commits December 15, 2025 18:13

Update platformio.ini

3de2c16

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge branch 'develop' into feature/ditch-arduino-httpclient

f752a17

Merge branch 'develop' into feature/ditch-arduino-httpclient

441688f

Experiement with global CA store

6b85a36

	OS_LOGE(TAG, "Failed to fetch hashes: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());
	OS_LOGE(TAG, "Failed to fetch hashes: %s [%u]", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

	OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());
	OS_LOGE(TAG, "Error while reading response: %s %d", HTTP::HTTPErrorToString(content.error), response.StatusCode());

	if (!Convert::ToUint32(value, seconds) \|\| seconds <= 0) {
	if (!Convert::ToUint32(value, seconds) \|\| seconds == 0) {

	SERPR_ERROR("Tried to read response from backend, but failed (%s), refusing to save domain to config", OpenShock::HTTP::HTTPErrorToString(response.Error()));
	SERPR_ERROR("Tried to read response from backend, but failed (%s), refusing to save domain to config", OpenShock::HTTP::HTTPErrorToString(content.error));

	OS_LOGE(TAG, "Failed to fetch list: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());
	OS_LOGE(TAG, "Failed to fetch list: %s [%u]", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

	OS_LOGE(TAG, "Failed to fetch firmware version: %s [%u] %s", HTTP::HTTPErrorToString(response.Error()), response.StatusCode(), content.data.c_str());
	OS_LOGE(TAG, "Failed to fetch firmware version: %s [%u]", HTTP::HTTPErrorToString(response.Error()), response.StatusCode());

refactor: Replace Arduino HTTP Client with esp native #391

Are you sure you want to change the base?

refactor: Replace Arduino HTTP Client with esp native #391

Uh oh!

Conversation

hhvrc commented Dec 4, 2025

Uh oh!

changeset-bot bot commented Dec 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hhvrc commented Dec 9, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Dec 17, 2025

Cpp-Linter Report ⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

changeset-bot bot commented Dec 4, 2025 •

edited

Loading