Feat/issue55 profile photo #392

zherujiang · 2025-02-18T09:17:19Z

Description

This PR adds a new edit profile photo feature. Prior to this, RecNet displays a user's Google account profile photo and does not allow changes. This feature allows any RecNet user to upload a profile photo, which is stored in an AWS S3 bucket and make updates to the user profile information in the database.

Frontend Changes

Modified /apps/recnet/src/components/setting/profile/ProfileEditForm.tsx adding new form element and function:
- Added profile photo <input> element with a photo preview image to accept and display the selected photo.
- Added handleUploadS3 function to get a secure uploadUrl from the backend, put objects to the S3 bucket, and update the form data.
- Modified the onSubmit function to call the handleUploadS3 function when a user saves the form.
- Updated the ProfileEditSchema, useForm hook to include the photoUrl field.
- Added a getUploadUrlMutation hook to send a request to the backend to get a secure S3 upload URL.
- Added new state hooks [isUploading, setIsUploading], [selectedFile, setSelectedFile], [photoPreviewUrl, setPhotoPreviewUrl] to manage the selected photo and upload status.
Created a new router getS3UploadUrl in the /apps/recnet/src/server/routers/user.ts
Modified the router updateUser in /apps/recnet/src/server/routers/user.ts to fetch the original photoUrl and delete the corresponding S3 file if a new profile photo is uploaded.

Backend Changes

Added a new folder s3 under /apps/recnet-api/src/modules
- Created a s3.controler.ts to handle requests to the s3url endpoint.
- Created a s3.service.ts. Included a getS3UploadUrl method and a deleteS3Object method.
- Created a s3.module.ts to package and export the s3 class.
Modified common.config.ts under /apps/recnet-api/src/config to register AWS S3 configuration.
Updated env.schema.ts under /apps/recnet-api/src/config` to include the s3 environment variable data types.

Other Changes

Updated app.module.ts under /apps/recnet-api/src to import and include the S3Module.
Updated package.json to include aws-sdk .
Install aws-sdk: @aws-sdk/client-s3 @aws-sdk/s3-request-presigner. Make sure to use aws-sdk v3.

How to Test

Log in to your account.
Navigate to the Profile page.
Click on the "Settings" button and open the first tab "Edit profile".
Click on "Select file" to choose a file and attach to the form.
Click on the "Save" button and wait for the upload to complete.
Check your profile photo and it should be updated to the new photo you uploaded.

Screenshots (if appropriate):

TODO

Add AWS S3 configurations in the .env file under /apps/recnet-api for the dev and prod environment for deployment

…tions

…leaking of s3 secrets

vercel · 2025-02-18T09:17:24Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
recnet	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 22, 2025 7:39am

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
recnet-docs	⬜️ Ignored (Inspect)	Visit Preview		Feb 22, 2025 7:39am

… update s3 module to use aws-sdk v3 syntax, remove unused file s3.ts

zherujiang · 2025-02-18T17:23:33Z

This PR has one failing check: lint/test/typecheck but on my local branch the lint checks have all passed. I took screnshots from the failed tests. Would this be because we haven't added the AWS S3 secrets in the .env file in the dev environment?
Can you take a look?

joannechen1223 · 2025-02-19T03:05:29Z

This PR has one failing check: lint/test/typecheck but on my local branch the lint checks have all passed. I took screnshots from the failed tests. Would this be because we haven't added the AWS S3 secrets in the .env file in the dev environment? Can you take a look?

Try to add the missing config in .env.test and commit again.

zherujiang · 2025-02-20T05:41:45Z

This PR has one failing check: lint/test/typecheck but on my local branch the lint checks have all passed. I took screnshots from the failed tests. Would this be because we haven't added the AWS S3 secrets in the .env file in the dev environment? Can you take a look?

Try to add the missing config in .env.test and commit again.

I could not add AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY to the file and commit the change because repository rule violations. Push can not contain secrets.

zherujiang · 2025-02-20T05:59:44Z

This PR has one failing check: lint/test/typecheck but on my local branch the lint checks have all passed. I took screnshots from the failed tests. Would this be because we haven't added the AWS S3 secrets in the .env file in the dev environment? Can you take a look?

Try to add the missing config in .env.test and commit again.

I'm trying to add the AWS secrets to the test environment the way Slack secrets are added. But I could not find where these env variables are in the code base. The only search result that contains these keywords is the .env.test. How do you add these variables?

joannechen1223 · 2025-02-20T06:57:03Z

@zherujiang https://docs.github.com/en/enterprise-cloud@latest/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-from-the-command-line#bypassing-push-protection

swh00tw · 2025-02-20T07:03:12Z

I think just add some random string as values for your new env variables in .env.test might solve the issue. From your screenshot, it seems like it's blocked bcz you are trying to push the secret (real value)?

zherujiang · 2025-02-20T07:28:51Z

I think just add some random string as values for your new env variables in .env.test might solve the issue. From your screenshot, it seems like it's blocked bcz you are trying to push the secret (real value)?

Thank you for the reply. I added fake secrets in .env.test and the checks are passed. I'll provide the real secrets separately for you to test on your local device.

zherujiang · 2025-02-20T07:29:07Z

@zherujiang https://docs.github.com/en/enterprise-cloud@latest/code-security/secret-scanning/working-with-secret-scanning-and-push-protection/working-with-push-protection-from-the-command-line#bypassing-push-protection

Thank you! I applied the setting and this worked.

swh00tw · 2025-02-20T17:16:31Z

One small thing, I think we should disable "save" button when we are saving.

swh00tw

LGTM. But I think the naming (files, functions, API endpoints) could be better. I think "s3" should not be part of the names, it sounds strongly coupled with AWS S3. What if in the future we decide to switch to another image storage service? we will need to rename everything. I think calling it photo.service.ts, photo.controller.ts, ... could be a bit better (you can come up with better name if you have).

And API is not really RESTful as well. Maybe change

GET /s3url to GET /photo/uploadLink since we are getting the url of the upload link
DELETE /s3url to DELETE /photo.

And I would recommend you document the API (request body, return, params) in recnet-api-model

swh00tw · 2025-02-20T17:22:21Z

apps/recnet/src/components/setting/profile/ProfileEditForm.tsx

@@ -315,7 +402,7 @@ export function ProfileEditForm() {
          type="submit"
          disabled={!formState.isValid}


I think we should disable when isUploading is true as well

Will make this change

joannechen1223

Left some comments. Please take a look, thank you!
Btw, as the flow of uploading photos is a bit complicated, it would be great if we could have a diagram to showcase how it works from end to end.

joannechen1223 · 2025-02-20T21:13:21Z

apps/recnet-api/.env.test

+
+# AWS S3
+export AWS_BUCKET_NAME="profile-photo-storage-dev"
+export AWS_ACCESS_KEY_ID="ABCD012FAKETESTSTRING"
+export AWS_SECRET_ACCESS_KEY="ABCDEFGHKB1BS3YTHISISAfakeTESTSECRET/pk"
+export AWS_BUCKET_REGION="us-east-2"


nit
AWS_BUCKET_NAME=test_aws_bucket_name
AWS_ACCESS_KEY_ID=test_aws_access_key_id
...

will make these changes.
I can make the diagram. Where do you want me to attach the diagram? In this PR or development handbook?

For the diagram, I think it would be great if you could create a page in Notion under Design Docs and fill in more details in the document.

joannechen1223 · 2025-02-20T21:14:29Z

apps/recnet-api/.env.test

-SLACK_TOKEN_ENCRYPTION_KEY=test_token_encryption_key
+SLACK_TOKEN_ENCRYPTION_KEY=test_token_encryption_key
+
+# AWS S3


Add these env var to env.sample

joannechen1223 · 2025-02-20T21:20:36Z

apps/recnet-api/src/modules/s3/s3.service.ts

+    this.s3Region = this.s3Config.s3Region;
+    this.s3BucketName = this.s3Config.bucketName;
+    this.accessKeyId = this.s3Config.accessKeyId;
+    this.secretAccessKey = this.s3Config.secretAccessKey;
+    this.s3 = new S3Client({
+      region: this.s3Region,
+      credentials: {
+        accessKeyId: this.accessKeyId,
+        secretAccessKey: this.secretAccessKey,
+      },
+    });


nit: We might not need to extract this from config again.

this.s3 = new s3Client({ region: this.s3Config.s3Region, credentials: { accessKeyId: this.s3Config.accessKeyId, secretAccessKey: this.s3Config.secretAccessKey, } })

joannechen1223 · 2025-02-20T21:23:02Z

apps/recnet-api/src/modules/s3/s3.service.ts

+    const timestamp = new Date()
+      .toLocaleString("en-US", {
+        year: "numeric",
+        month: "2-digit",
+        day: "2-digit",
+        hour: "2-digit",
+        minute: "2-digit",
+        second: "2-digit",
+        hour12: false,
+      })
+      .replace(/[/,: ]/g, "-");


Could you provide an example in the comment of this date format?

Yes. If the current date and time is February 21, 2025, 10:30:45 AM, the timestamp would be:
2025-02-21-10-30-45

Will also add the example to the code.

joannechen1223 · 2025-02-20T21:26:20Z

apps/recnet-api/src/modules/s3/s3.service.ts

+    } catch (error: unknown) {
+      const errorMessage =
+        error instanceof Error ? error.message : "Unknown error";
+      throw new Error(`Failed to delete S3 object: ${errorMessage}`);


Please define your error in recnet.error.const.ts and throw RecnetError to keep the error response consistent.

joannechen1223 · 2025-02-20T21:32:31Z

nit: Next time, kindly ensure that the screenshots do not contain Chinese text to maintain consistency and be inclusive to future maintainers.

Added AWS S3 config to .env.sample Changed AWS config and secret names in .env.test Added example time stamp to s3.service.ts Defined RecnetError for S3 upload and delete errors and throw RecnetError in s3.service.ts

…edesign API endpoints to meet RESTful requirements Rename s3.service.ts to photo-storage.service.ts Rename s3.controller.ts to photo-storage.controller.ts Rename s3.module.ts to photo-storage.module.ts Updated app.module.ts to import the PhotoStorageModule correctly Modified getUploadUrl router to generateUploadUrl and use the POST method instead, because this request results in the creation of a new resource -the S3 URL which will be used later to upload a new photo. Hence POST is the appropriate method. Updated the corresponding method generateS3UploadUrl in the backend photo-storage.service.ts and photo-storage.controller.ts, update to POST method.

…oading large file

… document the request and response schema for photo-storage API endpoints

joannechen1223

LGTM
Please remember to add your env vars to AWS before deployment

Reverts #392 I accidentally merged this PR to master instead of dev.

## Description This PR adds a new edit profile photo feature. Prior to this, RecNet displays a user's Google account profile photo and does not allow changes. This feature allows any RecNet user to upload a profile photo, which is stored in an AWS S3 bucket and make updates to the user profile information in the database. ## Related Issue * See [PR#392](#392) for previous comments and requested changes. Reopening PR for the same issue to merge to `dev` before `master`. ## Frontend Changes * Modified `/apps/recnet/src/components/setting/profile/ProfileEditForm.tsx` adding new form element and function: * Added profile photo `<input>` element with a photo preview image to accept and display the selected photo. * Added `handleUploadS3` function to get a secure uploadUrl from the backend, put objects to the S3 bucket, and update the form data. * Modified the `onSubmit` function to call the `handleUploadS3` function when a user saves the form. * Updated the `ProfileEditSchema`, `useForm` hook to include the photoUrl field. * Added a `generateUploadUrlMutation` hook to send a request to the backend to get a secure S3 upload URL. * Added new state hooks `[isUploading, setIsUploading]`, `[selectedFile, setSelectedFile]`, `[photoPreviewUrl, setPhotoPreviewUrl]` , `[fileError, setFileError]` to manage the selected photo and upload status. * Created a new router `generateS3UploadUrl` in the `/apps/recnet/src/server/routers/user.ts` * Modified the router `updateUser` in `/apps/recnet/src/server/routers/user.ts` to fetch the original photoUrl and delete the corresponding S3 file if a new profile photo is uploaded. ## Backend Changes * Added a new folder `photo-storage` under `/apps/recnet-api/src/modules` * Created a `photo-storage.controler.ts` to handle requests to the `photo-storage` endpoint. * Created a `photo-storage.service.ts`. Included a `generateS3UploadUrl` method and a `deleteS3Object` method. * Created a `photo-storage.module.ts` to package and export the s3 class. * Modified `common.config.ts` under `/apps/recnet-api/src/config` to register AWS S3 configuration. * Updated `env.schema.ts` under `/apps/recnet-api/src/config` to include the s3 environment variable data types. * Updated `app.module.ts` under `/apps/recnet-api/src` to include and import the `PhotoStorageModule`. ## Other Changes * Added `photo-storage.ts` under `/libs/recnet-api-model/src/lib/api` to document the new APIs related to `photo-storage`. * Updated `package.json` to include `aws-sdk/client-s3` and `aws-sdk/s3-request-presigner` . * Install aws-sdk: `@aws-sdk/client-s3 @aws-sdk/s3-request-presigner`. Make sure to use aws-sdk v3. ## How To Test 1. Log in to your account. 2. Navigate to the Profile page. 3. Click on the "Settings" button and open the first tab "Edit profile". 4. Click on "Choose file" to choose a file and attach to the form. 5. Click on the "Save" button and wait for the upload to complete. 6. Check your profile photo and it should be updated to the new photo you uploaded. ## Screenshots (if appropriate): <img width="863" alt="1_profile-photo-before" src="https://github.com/user-attachments/assets/608bfba6-883f-4fbd-a50f-329434d920f3" /> <img width="783" alt="2_edit-photo" src="https://github.com/user-attachments/assets/38ed449d-d572-4259-b7f6-fe42a4a44664" /> <img width="799" alt="3_photo-attached" src="https://github.com/user-attachments/assets/94b94372-5dfa-4c7d-9dba-5586850799f0" /> <img width="739" alt="4_uploading" src="https://github.com/user-attachments/assets/8c3bc13c-e06b-40d9-b254-5648e78bfd8f" /> <img width="851" alt="5_photo-updated" src="https://github.com/user-attachments/assets/a8267cfa-dd82-45ba-a6d6-11908eed4dcd" /> ## TODO - Add AWS S3 configurations in the `.env` file under `/apps/recnet-api` for the dev and prod environment for deployment

## RecNet auto-release action This is an auto-generated PR by recnet-release-action 🤖 Please make sure to test your changes in staging before merging. ## Related Issues - #392 - #371 ## Related PRs - #394 - #391 ## Staging links recnet-web: [https://vercel.live/link/recnet-git-dev-recnet-542617e7.vercel.app](https://vercel.live/link/recnet-git-dev-recnet-542617e7.vercel.app) recnet-api: [https://dev-api.recnet.io/api](https://dev-api.recnet.io/api)

## RecNet auto-release action This is an auto-generated PR by recnet-release-action 🤖 Please make sure to test your changes in staging before merging. Recnet-api version bump for deployment. Related to PR#390. ## Related Issues - #379 - #55 - #371 ## Related PRs - #382 - #391 - #392 - #394 - #390 ## Staging links recnet-web: [https://vercel.live/link/recnet-git-dev-recnet-542617e7.vercel.app](https://vercel.live/link/recnet-git-dev-recnet-542617e7.vercel.app) recnet-api: [https://dev-api.recnet.io/api](https://dev-api.recnet.io/api)

Zheru Jiang added 5 commits February 18, 2025 16:42

implemented issue55, change profile photo

bf35a9d

modify ProfileEditForm to upload profile photo only when submitting form

1745198

modify user router to only request uploadUrl when form is submitted

b2a2ffd

add delete profile photo from S3 feature. updated S3 bucket configura…

6f8249e

…tions

feat: edit profile photo, remove console.log in s3 module to prevent …

113959e

…leaking of s3 secrets

Zheru Jiang added 2 commits February 19, 2025 00:50

fix: fix lint issues

edab46b

fix: deprecate aws-sdk v2 (will be in maintenance mode Sept.2025) and…

1a17292

… update s3 module to use aws-sdk v3 syntax, remove unused file s3.ts

vercel bot deployed to Preview – recnet February 18, 2025 17:16 View deployment

vercel bot deployed to Preview – recnet-docs February 18, 2025 17:17 View deployment

zherujiang marked this pull request as ready for review February 18, 2025 17:20

zherujiang requested review from joannechen1223 and swh00tw February 18, 2025 17:20

fix: add fake AWS configuration for test purposes

a974e5c

swh00tw approved these changes Feb 20, 2025

View reviewed changes

joannechen1223 requested changes Feb 20, 2025

View reviewed changes

Zheru Jiang added 4 commits February 22, 2025 00:10

fix: add changes requested by reviewer

09f3b5b

Added AWS S3 config to .env.sample Changed AWS config and secret names in .env.test Added example time stamp to s3.service.ts Defined RecnetError for S3 upload and delete errors and throw RecnetError in s3.service.ts

fix: modify s3.service.ts to declare and extract s3Config once

21bf495

fix: Add file size limit of 3MB to ProfileEditForm.tsx to prevent upl…

e24f18d

…oading large file

fix: Added photo-storage.ts in 'libs/recnet-api-model/src/lib/api' to…

2f805dc

… document the request and response schema for photo-storage API endpoints

vercel bot deployed to Preview – recnet February 22, 2025 07:38 View deployment

joannechen1223 approved these changes Feb 24, 2025

View reviewed changes

zherujiang merged commit 157be46 into master Feb 24, 2025
4 checks passed

zherujiang mentioned this pull request Feb 24, 2025

Revert "Feat/issue55 profile photo" #393

Merged

zherujiang added a commit that referenced this pull request Feb 25, 2025

Revert "Feat/issue55 profile photo" (#393)

b3a6c60

Reverts #392 I accidentally merged this PR to master instead of dev.

zherujiang mentioned this pull request Feb 25, 2025

Feat/issue55 profile photo #394

Merged

github-actions bot mentioned this pull request Feb 26, 2025

Release dev to master #390

Merged

zherujiang mentioned this pull request Mar 11, 2025

Release dev to master #395

Merged

		@@ -315,7 +402,7 @@ export function ProfileEditForm() {
		type="submit"
		disabled={!formState.isValid}

Feat/issue55 profile photo #392

Feat/issue55 profile photo #392

Uh oh!

Conversation

zherujiang commented Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Frontend Changes

Backend Changes

Other Changes

How to Test

Screenshots (if appropriate):

TODO

Uh oh!

vercel bot commented Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zherujiang commented Feb 18, 2025

Uh oh!

joannechen1223 commented Feb 19, 2025

Uh oh!

zherujiang commented Feb 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zherujiang commented Feb 20, 2025

Uh oh!

joannechen1223 commented Feb 20, 2025

Uh oh!

swh00tw commented Feb 20, 2025

Uh oh!

zherujiang commented Feb 20, 2025

Uh oh!

zherujiang commented Feb 20, 2025

Uh oh!

swh00tw commented Feb 20, 2025

Uh oh!

swh00tw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joannechen1223 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joannechen1223 Feb 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joannechen1223 commented Feb 20, 2025

Uh oh!

joannechen1223 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zherujiang commented Feb 18, 2025 •

edited

Loading

vercel bot commented Feb 18, 2025 •

edited

Loading

zherujiang commented Feb 20, 2025 •

edited

Loading

joannechen1223 Feb 20, 2025 •

edited

Loading