Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ make clean-all # Also remove downloaded tools
```bash
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.0
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0
tools/protoc/osxm1/protoc *.proto --go-grpc_out=. --go_out=.

# Generate (adjust protoc path for your platform)
vermeer/tools/protoc/linux64/protoc vermeer/apps/protos/*.proto --go-grpc_out=vermeer/apps/protos/. --go_out=vermeer/apps/protos/. # please note remove license header if any
```

## Architecture
Expand Down Expand Up @@ -174,6 +176,7 @@ tools/protoc/osxm1/protoc *.proto --go-grpc_out=. --go_out=.
- Master scheduling: `vermeer/apps/master/tasks/tasks.go`
- Worker management: `vermeer/apps/master/workers/workers.go`
- HTTP endpoints: `vermeer/apps/master/services/http_master.go`
- Scheduler: `vermeer/apps/master/bl/scheduler_bl.go`

## Integration with HugeGraph

Expand Down Expand Up @@ -213,7 +216,7 @@ tools/protoc/osxm1/protoc *.proto --go-grpc_out=. --go_out=.
- Unit tests run in isolation without external dependencies

**Vermeer:**
- Test scripts in `vermeer/test/`
- Test scripts in `vermeer/test/`,with `vermeer_test.go` and `vermeer_test.sh`
- Configuration files in `vermeer/config/` (master.ini, worker.ini templates)

## CI/CD
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,12 +60,13 @@ graph TB
UI[Web UI Dashboard]
end

subgraph Master["Master Node :6688"]
HTTP[HTTP Server]
subgraph Master["Master Node"]
HTTP[HTTP Server :6688]
GRPC_M[gRPC Server :6689]
GM[Graph Manager]
TM[Task Manager]
WM[Worker Manager]
SCH[Scheduler]
end

subgraph Workers["Worker Nodes"]
Expand Down Expand Up @@ -145,6 +146,10 @@ For quick start and single-machine deployments, we recommend **Vermeer**:
# Pull the image
docker pull hugegraph/vermeer:latest

# Change config path in docker-compose.yml
volumes:
- ~/:/go/bin/config # Change here to your actual config path, e.g., vermeer/config

# Run with docker-compose
docker-compose up -d
```
Expand Down
9 changes: 7 additions & 2 deletions vermeer/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ go test -tags=vermeer_test -v -mode=scheduler
```bash
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.0
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0
tools/protoc/osxm1/protoc *.proto --go-grpc_out=. --go_out=.

# Generate (adjust protoc path for your platform)
vermeer/tools/protoc/linux64/protoc vermeer/apps/protos/*.proto --go-grpc_out=vermeer/apps/protos/. --go_out=vermeer/apps/protos/. # please note remove license header if any
```

## Architecture
Expand All @@ -81,6 +83,7 @@ vermeer/
│ │ ├── services/ # HTTP handlers
│ │ ├── workers/ # Worker management (WorkerManager, WorkerClient)
│ │ ├── tasks/ # Task scheduling
│ │ ├── schedules/ # Task scheduling strategies
│ │ └── graphs/ # Graph metadata management
│ ├── worker/ # Worker service entry
│ ├── compute/ # Worker-side compute logic
Expand Down Expand Up @@ -146,6 +149,7 @@ Algorithms implement the interface defined in `apps/compute/api.go`. Each algori
- Master scheduling: `apps/master/tasks/tasks.go`
- Worker management: `apps/master/workers/workers.go`
- HTTP endpoints: `apps/master/services/http_master.go`
- Scheduler: `vermeer/apps/master/bl/scheduler_bl.go`

## Development Workflow

Expand All @@ -167,7 +171,8 @@ Algorithms implement the interface defined in `apps/compute/api.go`. Each algori
1. Edit `.proto` files in `apps/protos/`
2. Regenerate Go code using protoc (adjust path for platform):
```bash
tools/protoc/osxm1/protoc apps/protos/*.proto --go-grpc_out=. --go_out=.
# Generate (adjust protoc path for your platform)
vermeer/tools/protoc/linux64/protoc vermeer/apps/protos/*.proto --go-grpc_out=vermeer/apps/protos/. --go_out=vermeer/apps/protos/. # please note remove license header if any
```

## Configuration
Expand Down
10 changes: 8 additions & 2 deletions vermeer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ vermeer/
│ ├── master/ # Master service
│ │ ├── services/ # HTTP handlers
│ │ ├── workers/ # Worker management
| | ├── schedules/ # Task scheduling strategies
│ │ └── tasks/ # Task scheduling
│ ├── compute/ # Worker-side compute logic
│ ├── graphio/ # Graph I/O (HugeGraph, CSV, HDFS)
Expand Down Expand Up @@ -117,7 +118,12 @@ docker run -v ~/vermeer-config:/go/bin/config hugegraph/vermeer --env=worker

#### Docker Compose

Update `master_peer` in `~/worker.ini` to `172.20.0.10:6689`, then:
Update `master_peer` in `~/worker.ini` to `172.20.0.10:6689`, and edit ``docker-compose.yml`` to mount your config directory:

```yaml
volumes:
- ~/:/go/bin/config # Change here to your actual config path
```

```bash
docker-compose up -d
Expand Down Expand Up @@ -479,7 +485,7 @@ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.0
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0

# Generate (adjust protoc path for your platform)
tools/protoc/osxm1/protoc *.proto --go-grpc_out=. --go_out=.
vermeer/tools/protoc/linux64/protoc vermeer/apps/protos/*.proto --go-grpc_out=vermeer/apps/protos/. --go_out=vermeer/apps/protos/. # please note remove license header if any
```

## Performance Tuning
Expand Down
Loading