Add Memory Usage Tracker For Programs and Update Tests

[Abhinav-Prajapati]

User description

Overview

This PR introduces a memory tracking feature for all programs by leveraging the /usr/bin/time command. The goal is to monitor and validate how much memory each programming language is using during execution, and to align memory usage reporting across languages (C, Python, Node.js, etc.).

---

Changes

Memory Tracking Feature

• Implemented memory usage tracking using /usr/bin/time for accurate and consistent memory measurement across languages.

Expanded Test Cases

• Added and updated C test cases in testJson.js to cover:
  • Heap Allocation Test: Allocates 50MB on the heap, touches the memory, then frees it. Verifies both output and memory usage.
  • Stack Allocation Test: Recursively allocates 1MB per call to reach 6MB of stack usage, ensuring it remains within safe system limits.
• Introduced an approxMemoryUses field for test cases to define expected memory ranges.

Code Improvements

• Updated all test case program strings to use template literals (backticks) instead of string concatenation for improved readability and maintainability.

---

PR Type

Enhancement, Tests

---

Description

• Add memory usage tracking using /usr/bin/time command
• Implement memory validation in test cases with tolerance checks
• Convert test scripts to template literals for better readability
• Add new C test cases for heap and stack memory allocation

---

Changes walkthrough 📝

	Relevant files
Enhancement	code.service.js Add memory tracking to code execution                                        services/code.service.js • Integrate /usr/bin/time command for memory monitoring during code execution • Parse memory report from /tmp/memory_report.txt and adjust for Alpine Linux • Add memory field to response object with KB to adjusted units conversion • Minor code formatting improvements (spacing and function declarations) +26/-7
Tests	testJson.js Update test cases with memory expectations                              tests/data/testJson.js • Convert all test script strings from concatenation to template literals • Add approxMemoryUses field to existing test cases with expected memory values • Add two new C test cases for heap allocation (50MB) and stack allocation (6MB) • Update stdin handling and script formatting across all language tests +164/-91 test.js Add memory validation to test assertions                                  tests/test.js • Add memory tolerance validation with 20% tolerance threshold • Implement memory usage assertions comparing actual vs expected values • Add MEMORY_TOLERANCE_PERCENT constant for configurable tolerance +11/-0

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[gemini-code-assist]

Summary of Changes

Hello @Abhinav-Prajapati, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request adds a crucial feature to track and report the memory usage of executed programs using /usr/bin/time. It also enhances the test suite with new memory-specific tests for C and adds memory validation checks for existing language tests, ensuring better monitoring and control over program resource consumption.

Highlights

• Memory Usage Tracking: Implemented memory usage tracking for program executions by leveraging the /usr/bin/time command to capture the maximum resident set size (memory usage) and store it in a temporary file (/tmp/memory_report.txt). This data is then read and included in the API response.
• New C Memory Tests: Added two new C test cases specifically designed to validate memory usage: one for allocating and touching 50MB on the heap, and another for recursive stack allocation up to 6MB.
• Memory Validation in Tests: Introduced an approxMemoryUses field in test cases to specify expected memory ranges. The test suite now includes assertions to check the actual memory reported by the service against these expected values within a defined tolerance (20%).
• Code Readability: Refactored test case scripts in testJson.js to use template literals instead of string concatenation, improving readability and maintainability.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configureGemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[qodo-free-for-open-source-projects]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Hardcoded Division The memory calculation uses a hardcoded division by 4 for Alpine Linux adjustment without proper OS detection or configuration. This could lead to incorrect memory reporting on non-Alpine systems. response.memory = memoryKB ? memoryKB / 4 : null console.log('memory used', response.memory) Error Handling Memory report reading has basic error handling but continues execution silently when memory tracking fails. This could mask important monitoring failures in production. let memoryKB = null try { const path = '/tmp/memory_report.txt' await fs.promises.access(path, fs.constants.F_OK) const memoryReport = await fs.promises.readFile(path, 'utf8') memoryKB = parseInt(memoryReport.trim(), 10) } catch (err) { console.warn(`Memory report not found or failed to read: ${err.message}`) } // Adjust if you're on Alpine to divide by 4 response.memory = memoryKB ? memoryKB / 4 : null console.log('memory used', response.memory) Memory Values The hardcoded approxMemoryUses values may not be portable across different environments and could cause test failures on systems with different memory characteristics or configurations. approxMemoryUses: 2744, status: 200, error: 0, }, }, { name: 'cpp : print stdin', reqObject: { language: 'cpp', script: ` #include<bits/stdc++.h> using namespace std; int main(){ int a; while(cin >> a){ cout << a << endl; } return 0; }`, stdin: '1 2 3', }, expectedResponse: { val: '1\n2\n3\n', approxMemoryUses: 2680, status: 200, error: 0, }, }, { name: 'nodejs : hello world', reqObject: { language: 'nodejs', script: `console.log('hello world')`, }, expectedResponse: { val: 'hello world\n', approxMemoryUses: 44540, status: 200, error: 0, }, }, { name: 'nodejs : print stdin', reqObject: { language: 'nodejs', script: ` process.stdin.setEncoding('utf8'); process.stdin.on('data', (input) => { console.log(input.trim()); });`, stdin: '1 2 3', }, expectedResponse: { val: '1 2 3\n', approxMemoryUses: 44888, status: 200, error: 0, }, }, { name: 'python : hello world', reqObject: { language: 'python', script: `print('hello world')`, }, expectedResponse: { val: 'hello world\n', approxMemoryUses: 5544, status: 200, error: 0, }, }, { name: 'python : print stdin', reqObject: { language: 'python', script: ` import sys for line in sys.stdin: print(line.strip())`, stdin: '1 2 3', }, expectedResponse: { val: '1 2 3\n', approxMemoryUses: 5800, status: 200, error: 0, }, }, { name: 'c : hello world', reqObject: { language: 'c', script: `#include <stdio.h> int main(){ printf("hello world"); return 0; } `, }, expectedResponse: { val: 'hello world', approxMemoryUses: 900, status: 200, error: 0, }, }, { name: 'c : print stdin', reqObject: { language: 'c', script:` #include <stdio.h> int main() { int number; while (scanf("%d", &number) == 1) { printf("%d\\n", number); } return 0; } `, stdin: '1 2 3', }, expectedResponse: { val: '1\n2\n3\n', approxMemoryUses: 924, status: 200, error: 0, }, }, { name: 'java : hello world', reqObject: { language: 'java', script: ` import java.util.Scanner; public class Solution { public static void main(String[] args) { System.out.println("hello world"); } }`, }, expectedResponse: { val: 'hello world\n', approxMemoryUses: 33000, status: 200, error: 0, }, }, { name: 'java : print stdin', reqObject: { language: 'java', script: ` import java.util.Scanner; public class Solution { public static void main(String[] args) { Scanner scanner = new Scanner(System.in); while (scanner.hasNextInt()) { int number = scanner.nextInt(); System.out.println(number); } scanner.close(); } } `, stdin: '1 2 3', }, expectedResponse: { val: '1\n2\n3\n', approxMemoryUses: 35900, status: 200, error: 0, }, }, { name: 'ruby : print hello world', reqObject: { language: 'ruby', script: `print "hello world"`, }, expectedResponse: { val: 'hello world', approxMemoryUses: 22800, status: 200, error: 0, }, }, { name: 'ruby : print stdin', reqObject: { language: 'ruby', script: ` user_input = gets.chomp puts user_input`, stdin: '10', }, expectedResponse: { val: '10\n', approxMemoryUses: 22800, status: 200, error: 0, }, }, { name: 'TLE test', reqObject: { language: 'nodejs', script: `for(let i = 0 ;; ) { i++ } `, }, expectedResponse: { val: 'Time limit exceeded', status: 200, error: 1, }, }, { name: 'MLE test', reqObject: { language: 'python', script: `one_gb_data = bytearray(1000 * 1024 * 1024)`, }, expectedResponse: { val: 'Memory limit exceeded', status: 200, error: 1, }, }, { name: 'MLE test 2', reqObject: { language: 'python', script: ` import time def consume_memory(target_mb, duration_sec): float_size = 8 floats_per_mb = (1024 * 1024) // float_size total_floats = target_mb * floats_per_mb iterations = int(duration_sec / 0.1) floats_per_iteration = total_floats // iterations memory_hog = [] for _ in range(iterations): memory_hog.extend([0.0] * floats_per_iteration) time.sleep(0.1) consume_memory(1000, 1)`, }, expectedResponse: { val: 'Memory limit exceeded', status: 200, error: 1, }, }, { name: 'MLE test 3', reqObject: { language: 'python', script: ` a = [100] for i in a: a.append(i) `, }, expectedResponse: { val: 'Memory limit exceeded', status: 200, error: 1, }, }, { name: 'OPEN AI test promptv1', reqObject: { language: 'promptv1', prompt: 'The question is what is 2 plus 2. The answer given is 4.', }, expectedResponse: { val: {}, status: 200, error: 0, }, }, { name: 'OPEN AI test promptv2', reqObject: { language: 'promptv2', prompt: 'The question is what is 2 plus 2. The answer given is 4.', }, expectedResponse: { val: {}, status: 200, error: 0, }, }, { name: 'c :Heap memory allocation', reqObject: { language: 'c', script: ` #include <stdio.h> #include <stdlib.h> int main() { size_t memory_size = 50 * 1024 * 1024; char *memory_block = malloc(memory_size); if (memory_block == NULL) { printf("Failed to allocate memory\\n"); return 1; } printf("Memory allocation done\\n"); for (size_t i = 0; i < memory_size; i += 4096) { memory_block[i] = (char)(i % 256); } printf("Memory touched and initialized\\n"); free(memory_block); printf("Memory freed\\n"); return 0; } `, }, expectedResponse: { val: 'Memory allocation done\nMemory touched and initialized\nMemory freed\n', status: 200, error: 0, approxMemoryUses: 51000, }, }, { name: 'c :Stack memory allocation', reqObject: { language: 'c', script: ` #include <stdio.h> #include <string.h> #define ONE_MB (1024 * 1024) void stack_allocate(int remaining_bytes, int depth) { if (remaining_bytes <= 0) { printf("Memory allocated on stack\\n"); return; } char buffer[ONE_MB]; memset(buffer, 0, ONE_MB); // Touch the memory stack_allocate(remaining_bytes - ONE_MB, depth + 1); } int main() { stack_allocate(6 * ONE_MB, 0); return 0; } `, }, expectedResponse: { val: 'Memory allocated on stack\n', status: 200, error: 0, approxMemoryUses: 6500, // Max stack uses limit is ~10 MB },

[Abhinav-Prajapati]

/gemini review

[qodo-free-for-open-source-projects]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	✅ Make Alpine memory adjustment configurable Suggestion Impact: The commit implemented the core suggestion by adding OS detection for Alpine Linux and conditionally applying the memory adjustment, though with a simpler implementation than suggested code diff: + const isAlpine = fs.existsSync('/etc/alpine-release'); + + if (memoryKB) { + response.memory = isAlpine ? memoryKB / 4 : memoryKB; + } else { + response.memory = null; + } The hardcoded division by 4 for Alpine Linux is fragile and may not work correctly on other systems. Consider detecting the OS type or making this configurable through environment variables to ensure accurate memory reporting across different environments. services/code.service.js [260-261] -// Adjust if you're on Alpine to divide by 4 -response.memory = memoryKB ? memoryKB / 4 : null +// Adjust memory based on OS type (Alpine Linux reports in different units) +const isAlpine = process.env.OS_TYPE === 'alpine' || fs.existsSync('/etc/alpine-release') +response.memory = memoryKB ? (isAlpine ? memoryKB / 4 : memoryKB) : null [Suggestion processed] Suggestion importance[1-10]: 9 __ Why: The current implementation unconditionally divides the reported memory by 4, which is only correct for Alpine Linux as stated in the code comment. This will cause incorrect memory reporting on any other operating system. The suggestion provides a robust fix by detecting the OS before applying the adjustment, which is critical for the feature's correctness.	High
	Check time command availability before usage The /usr/bin/time command may not be available on all systems or Docker images. Consider checking for its existence before using it, or provide a fallback mechanism to ensure the code execution doesn't fail when the command is unavailable. services/code.service.js [234] -command = `cd /tmp/ && /usr/bin/time -f "%M" -o /tmp/memory_report.txt timeout ${langConfig.timeout}s ${langConfig.run}` +const timeCmd = fs.existsSync('/usr/bin/time') ? '/usr/bin/time -f "%M" -o /tmp/memory_report.txt' : '' +command = `cd /tmp/ && ${timeCmd} timeout ${langConfig.timeout}s ${langConfig.run}` Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly points out that hardcoding the path to /usr/bin/time can cause the application to fail in environments where the utility is missing or located elsewhere. Providing a fallback mechanism by checking for the command's existence significantly improves the portability and robustness of the code execution feature.	Medium
	Add validation for parsed memory values The parseInt function should include error handling for invalid numeric values. If the memory report contains non-numeric data, parseInt will return NaN, which could cause issues downstream when performing arithmetic operations. services/code.service.js [250-258] let memoryKB = null try { const path = '/tmp/memory_report.txt' await fs.promises.access(path, fs.constants.F_OK) const memoryReport = await fs.promises.readFile(path, 'utf8') - memoryKB = parseInt(memoryReport.trim(), 10) + const parsed = parseInt(memoryReport.trim(), 10) + memoryKB = isNaN(parsed) ? null : parsed } catch (err) { console.warn(`Memory report not found or failed to read: ${err.message}`) } Apply / Chat Suggestion importance[1-10]: 6 __ Why: The suggestion correctly identifies that parseInt can return NaN if the memory report is malformed. While the current code might handle this implicitly because NaN is falsy, explicitly checking with isNaN makes the code more robust and clearer, improving error handling and maintainability.	Low
Update

[gemini-code-assist]

Code Review

This pull request introduces memory tracking for programs using /usr/bin/time and updates test cases to validate memory usage. The changes include integrating memory monitoring into the code execution service, adding new C test cases for heap and stack allocation, and updating test assertions with memory tolerance validation. The code has been improved by converting test scripts to template literals for better readability.

[gemini-code-assist]

Code Review

This pull request introduces memory tracking for programs and updates tests to validate memory usage. The changes include integrating /usr/bin/time for memory monitoring, adding memory validation to test cases, and converting test scripts to template literals. The addition of memory tolerance validation and new C test cases for heap and stack memory allocation enhances the testing suite.

[gemini-code-assist]

Consider adding .trim() to the input before logging it, to avoid any unexpected whitespace in the output.

console.log(input.trim());

[qodo-code-review]

You are above your monthly Qodo Merge usage quota. For more information, please visit here.

[qodo-code-review]

You are above your monthly Qodo Merge usage quota. For more information, please visit here.

[qodo-code-review]

You are above your monthly Qodo Merge usage quota. For more information, please visit here.

[singodiyashubham87]

Please resolve the requested changes.

[singodiyashubham87]

After making the changes, Please make sure to test everything.

[singodiyashubham87]

LGTM!

Please wait for approval from @shashanksingh2002 before merging.