This repository was archived by the owner on Dec 1, 2025. It is now read-only.
forked from delta-io/delta-sharing
-
Notifications
You must be signed in to change notification settings - Fork 0
Persistence of shares implemented(not yet a full version) #219
Draft
basicaleksa1
wants to merge
1
commit into
main
Choose a base branch
from
persistent-storage-manager
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| plugins { | ||
| `java-library` | ||
| id("whitefox.java-conventions") | ||
| } | ||
|
|
||
| val quarkusPlatformGroupId: String by project | ||
| val quarkusPlatformArtifactId: String by project | ||
| val quarkusPlatformVersion: String by project | ||
|
|
||
| repositories { | ||
| mavenCentral() | ||
| } | ||
|
|
||
| dependencies { | ||
| implementation(enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")) | ||
|
|
||
| implementation("io.quarkus:quarkus-jdbc-postgresql") | ||
| implementation("io.quarkus:quarkus-hibernate-orm-panache") | ||
|
|
||
|
|
||
| // QUARKUS | ||
| compileOnly("jakarta.enterprise:jakarta.enterprise.cdi-api") | ||
| compileOnly("jakarta.ws.rs:jakarta.ws.rs-api") | ||
| compileOnly("org.eclipse.microprofile.config:microprofile-config-api") | ||
|
|
||
| compileOnly(project(":server:core")) | ||
|
|
||
| // TEST | ||
| testImplementation("io.quarkus:quarkus-junit5") | ||
| testImplementation("io.quarkus:quarkus-arc") | ||
| } | ||
|
|
||
|
|
||
| tasks.withType<JavaCompile> { | ||
| options.encoding = "UTF-8" | ||
| options.compilerArgs.add("-parameters") | ||
| } | ||
|
|
||
| tasks.test { | ||
| useJUnitPlatform() | ||
| } |
118 changes: 118 additions & 0 deletions
118
server/persistence/jdbc/src/main/java/JdbcStorageManager.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| import io.whitefox.core.*; | ||
| import io.whitefox.persistence.StorageManager; | ||
| import io.whitefox.persistence.exceptions.InvalidPageTokenException; | ||
| import jakarta.enterprise.context.ApplicationScoped; | ||
| import jakarta.inject.Inject; | ||
| import jakarta.transaction.Transactional; | ||
| import mapper.ShareMapper; | ||
| import repository.ShareRepository; | ||
|
|
||
| import java.util.List; | ||
| import java.util.Optional; | ||
| import java.util.stream.Collectors; | ||
|
|
||
| @Transactional | ||
| @ApplicationScoped | ||
| public class JdbcStorageManager implements StorageManager { | ||
|
|
||
| @Inject | ||
| ShareRepository shareRepository; | ||
|
|
||
| @Inject | ||
| public JdbcStorageManager(){ | ||
| this.shareRepository = new ShareRepository(); | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<Share> getShare(String share) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public ResultAndTotalSize<List<Share>> getShares(int offset, int maxResultSize) { | ||
|
|
||
| List<Share> shares = shareRepository.listAll().stream() | ||
| .map(ShareMapper::shareDaoToShare) | ||
| .collect(Collectors.toList()); | ||
| var totalSize = shares.size(); | ||
| if (offset > totalSize) { | ||
| throw new InvalidPageTokenException( | ||
| String.format("Invalid Next Page Token: token %s is larger than totalSize", offset)); | ||
| } else { | ||
| return new ResultAndTotalSize<>( | ||
| shares.stream().skip(offset).limit(maxResultSize).collect(Collectors.toList()), | ||
| totalSize); | ||
| } | ||
|
Comment on lines
+34
to
+45
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This eagerly fetches all shares, it might get very slow when a lot of shares are created. |
||
| } | ||
|
|
||
| @Override | ||
| public Optional<SharedTable> getSharedTable(String share, String schema, String table) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<ResultAndTotalSize<List<Schema>>> listSchemas(String share, int offset, int maxResultSize) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<ResultAndTotalSize<List<SharedTable>>> listTables(String share, String schema, int offset, int maxResultSize) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<ResultAndTotalSize<List<SharedTable>>> listTablesOfShare(String share, int offset, int finalMaxResults) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public Metastore createMetastore(Metastore metastore) { | ||
| return null; | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<Metastore> getMetastore(String name) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public Storage createStorage(Storage storage) { | ||
| return null; | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<Storage> getStorage(String name) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public Provider createProvider(Provider provider) { | ||
| return null; | ||
| } | ||
|
|
||
| @Override | ||
| public Optional<Provider> getProvider(String name) { | ||
| return Optional.empty(); | ||
| } | ||
|
|
||
| @Override | ||
| public InternalTable createInternalTable(InternalTable internalTable) { | ||
| return null; | ||
| } | ||
|
|
||
| @Override | ||
| public Share createShare(Share share) { | ||
| shareRepository.persist(ShareMapper.shareToShareDao(share)); | ||
| return share; | ||
| } | ||
|
|
||
| @Override | ||
| public Share updateShare(Share newShare) { | ||
| return null; | ||
| } | ||
|
|
||
| @Override | ||
| public Share addTableToSchema(Share shareObj, Schema schemaObj, Provider providerObj, InternalTable table, SharedTableName sharedTableName, Principal currentUser, long millis) { | ||
| return null; | ||
| } | ||
| } | ||
39 changes: 39 additions & 0 deletions
39
server/persistence/jdbc/src/main/java/mapper/ShareMapper.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| package mapper; | ||
|
|
||
| import io.whitefox.core.Principal; | ||
| import io.whitefox.core.Schema; | ||
| import io.whitefox.core.Share; | ||
| import jakarta.inject.Singleton; | ||
| import model.ShareDao; | ||
|
|
||
| import java.util.Map; | ||
| import java.util.Optional; | ||
| import java.util.Set; | ||
|
|
||
| @Singleton | ||
| public class ShareMapper { | ||
|
|
||
| public static Share shareDaoToShare(ShareDao shareDao){ | ||
| String id = shareDao.getId(); | ||
| String name = shareDao.getName(); | ||
| Optional<String> comment = shareDao.getComment(); | ||
| Map<String, Schema> schemas = Map.of(); | ||
| Set<Principal> recipients = Set.of(); | ||
| Long createdAt = shareDao.getCreatedAt(); | ||
| Principal principal = new Principal(shareDao.getCreatedBy()); | ||
| Long updatedAt = shareDao.getUpdatedAt(); | ||
| return new Share(name, id, schemas, comment, recipients, createdAt, principal, updatedAt, principal, principal ); | ||
| } | ||
|
|
||
| public static ShareDao shareToShareDao(Share share){ | ||
| String id = share.id(); | ||
| String name = share.name(); | ||
| String comment = share.comment().orElse(""); | ||
| Long createdAt = share.createdAt(); | ||
| String createdBy = share.createdBy().name(); | ||
| Long updatedAt = share.updatedAt(); | ||
| String updatedBy = share.updatedBy().name(); | ||
| String owner = share.owner().name(); | ||
| return new ShareDao(id, name, comment, createdAt, createdBy, updatedAt, updatedBy, owner); | ||
| } | ||
| } |
103 changes: 103 additions & 0 deletions
103
server/persistence/jdbc/src/main/java/model/ShareDao.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| package model; | ||
|
|
||
| import jakarta.persistence.*; | ||
|
|
||
| import java.util.Optional; | ||
| @Entity | ||
| @Table(name = "share") | ||
| public class ShareDao { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you can/want you can use Lombok to auto-generate some boilerplate |
||
|
|
||
| @Id | ||
| private String id; | ||
| private String name; | ||
|
|
||
| private String comment; | ||
| //todo make these fields | ||
| // private Map<String, String> schema; | ||
| // private Set<PrincipalDAO> recipients; | ||
| private Long createdAt; | ||
|
|
||
| private String createdBy; | ||
| private Long updatedAt; | ||
|
|
||
| private String updatedBy; | ||
|
|
||
| private String owner; | ||
|
|
||
| public ShareDao() { | ||
| } | ||
|
|
||
| public ShareDao(String id, String name, String comment, Long createdAt, String createdBy, Long updatedAt, String updatedBy, String owner) { | ||
| this.id = id; | ||
| this.name = name; | ||
| this.comment = comment; | ||
| this.createdAt = createdAt; | ||
| this.createdBy = createdBy; | ||
| this.updatedAt = updatedAt; | ||
| this.updatedBy = updatedBy; | ||
| this.owner = owner; | ||
| } | ||
|
|
||
| public String getId() { | ||
| return id; | ||
| } | ||
|
|
||
| public void setId(String id) { | ||
| this.id = id; | ||
| } | ||
|
|
||
| public String getName() { | ||
| return name; | ||
| } | ||
|
|
||
| public Optional<String> getComment(){ | ||
| return Optional.ofNullable(this.comment); | ||
| } | ||
| public void setComment(String comment){ | ||
| this.comment = comment; | ||
| } | ||
|
|
||
| public void setName(String name) { | ||
| this.name = name; | ||
| } | ||
|
|
||
| public Long getCreatedAt() { | ||
| return createdAt; | ||
| } | ||
|
|
||
| public void setCreatedAt(Long createdAt) { | ||
| this.createdAt = createdAt; | ||
| } | ||
|
|
||
| public String getCreatedBy() { | ||
| return createdBy; | ||
| } | ||
|
|
||
| public void setCreatedBy(String createdBy) { | ||
| this.createdBy = createdBy; | ||
| } | ||
|
|
||
| public Long getUpdatedAt() { | ||
| return updatedAt; | ||
| } | ||
|
|
||
| public void setUpdatedAt(Long updatedAt) { | ||
| this.updatedAt = updatedAt; | ||
| } | ||
|
|
||
| public String getUpdatedBy() { | ||
| return updatedBy; | ||
| } | ||
|
|
||
| public void setUpdatedBy(String updatedBy) { | ||
| this.updatedBy = updatedBy; | ||
| } | ||
|
|
||
| public String getOwner() { | ||
| return owner; | ||
| } | ||
|
|
||
| public void setOwner(String owner) { | ||
| this.owner = owner; | ||
| } | ||
| } | ||
14 changes: 14 additions & 0 deletions
14
server/persistence/jdbc/src/main/java/repository/ShareRepository.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| package repository; | ||
|
|
||
| import io.quarkus.hibernate.orm.panache.PanacheRepositoryBase; | ||
| import jakarta.enterprise.context.ApplicationScoped; | ||
| import model.ShareDao; | ||
|
|
||
| @ApplicationScoped | ||
| public class ShareRepository implements PanacheRepositoryBase<ShareDao, String> { | ||
|
|
||
| @Override | ||
| public void persist(ShareDao shareDao) { | ||
| PanacheRepositoryBase.super.persist(shareDao); | ||
| } | ||
| } |
Empty file.
6 changes: 6 additions & 0 deletions
6
server/persistence/jdbc/src/main/resources/application.properties
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| quarkus.datasource.db-kind=postgresql | ||
| quarkus.datasource.username=postgres | ||
| quarkus.datasource.password=postgres | ||
| quarkus.datasource.reactive.url=jdbc:postgresql://localhost:5432/postgres | ||
| quarkus.hibernate-orm.database.generation=drop-and-create | ||
| quarkus.hibernate-orm.active=true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's use constructor based injection, also have a look at:
https://agile-lab-dev.github.io/whitefox/docs/development_guidelines