SDAP-118 create a new ranking module #31
Conversation
initial work
|
Can one of the admins verify this patch? |
lewismc
left a comment
lewismc
left a comment
There was a problem hiding this comment.
There are several comments.
Please make sure that the files are removed from core if they are being built in to ranking.
This is looking good.
| @@ -0,0 +1,215 @@ | |||
| <?xml version="1.0" encoding="UTF-8"?> | |||
There was a problem hiding this comment.
Please ensure that license header is formatted correctly.
Also ensure that the code formatting matches what is present in the other pom.xml files.
There was a problem hiding this comment.
Some files will be moved to ranking module in the next step, such as RankingTrainData.java because I worked on the expert provided train data now.
| </resource> | ||
| </resources> | ||
|
|
||
| <plugins> |
There was a problem hiding this comment.
We do not need the appassembler-maven-plugin configuration
| </plugin> | ||
|
|
||
| <plugin> | ||
| <groupId>org.apache.maven.plugins</groupId> |
There was a problem hiding this comment.
We do not need the maven-shade-plugin configuration
ranking/src/main/assembly/bin.xml
Outdated
| @@ -0,0 +1,50 @@ | |||
| <?xml version="1.0" encoding="UTF-8"?> | |||
| @@ -0,0 +1,57 @@ | |||
| /* | |||
There was a problem hiding this comment.
Make sure that code formatting is 2 space indents
|
|
||
| public static void main(String[] args) { | ||
| SparkFormatter sf = new SparkFormatter(); | ||
| sf.toSparkSVMformat("C:/mudrodCoreTestData/rankingResults/inputDataForSVM.csv", "C:/mudrodCoreTestData/rankingResults/inputDataForSVM_spark.txt"); |
| * into a user specified directory. | ||
| */ | ||
| public void process() { | ||
| public void convert2TrainSet() { |
There was a problem hiding this comment.
Change convert2TrainSet to convertToTrainSet
| for (int i = 1; i < arr.length - row; i++) { | ||
| List<String> colList = new ArrayList<String>(); // create vector to store all values inside of a column, which is stored inside 2D vector | ||
| for (int col = 0; col < arr[0].length - 1; col++) // Columns go until the next to last column | ||
| for (int col = 1; col < arr[0].length - 2; col++) // Columns go until the next to last column |
There was a problem hiding this comment.
Before Yongyao deleted the first column of the experts provided data manually and then invoked the function to process data.
|
|
||
| public RankTrainDataFactory(Properties props, ESDriver es, SparkDriver spark) { | ||
| super(props, es, spark); | ||
| // TODO Auto-generated constructor stub |
| } | ||
|
|
||
| // start session | ||
| // mode: overwrite or append |
|
Also @quintinali please make sure that you test this code with monthly input logs. |
|
@quintinali can you update the PR with the changes ? Thanks |
3 participants
No description provided.