Planning to clean up on amber directory

[Yicong-Huang]

I want to start an effort to clean up the amber directory. Currently it contains TexeraWebServer, CUDriver, and CUWorker codebase. I want to separate them and move codebase to corresponding sub modules. We can also clean up dependencies during this process. @bobbai00 as you previously moved out many mirco-services out from TexeraWebServer, can you give a summary of the current status, so we can estimate and make a plan?

[chenlica]

It will be helpful to include the tree structure of the current folder, and possibly our internal system architecture.

[bobbai00]

Here is the current root level folders along with the underneath micro services:

⎿  .
├── access-control-service (AccessControlService)
├── amber (ComputingUnitMaster, ComputingUnitWorker, TexeraWebApplication)
├── computing-unit-managing-service (ComputingUnitManagingService)
├── config-service (ConfigService)
├── file-service (FileService)
├── pyright-language-service (PyrightLanguageService)
└── workflow-compiling-service (WorkflowCompilingService)

Major TODOs are indeed on cleaning up the amber folder, possibly coming up with a separate folder for each micro service underneath amber.

[Yicong-Huang]

Let's focus on internal of amber folder. We don't need to discuss any services that are already extracted out.

[Yicong-Huang]

Can you explain the internal structure of amber, on the high level?

[bobbai00]

Sure, at high level, amber folder contains code in the following two packages:

• org.apache.amber: main logic of the amber engine, 3 child packages are underneath:
  • clustering
  • engine
  • error
• org.apache.texera: webserver related logic. 2 child packages are underneath:
  • web: ComputingUnitMaster, ComputingUnitWorker, TexeraWebApplication are here
  • workflow: duplicated definitions of WorkflowCompiler, LogicalLink and LogicalPlan (org.apache.amber.compiler.model contain another copy)

[chenlica]

Should we rename org.apache.amber to org.apache.texera.amber?

[Yicong-Huang]

Yes I agree. All code should be under Texera namespace. If no objection I can create an issue