Skip to content

Upgrade SnakeYAML #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
desyncr wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Upgrade SnakeYAML #16

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
d6820dc
fix generic
Olezha Dec 10, 2019
2f4d210
explanatory comment added
Olezha Dec 10, 2019
86a7d0a
update java version to 8 and snakeyaml to 1.25
Olezha Dec 10, 2019
c5b130d
bypass for termTable
Olezha Dec 10, 2019
7ea5f86
clean YamlReaderWriter and test
Olezha Dec 10, 2019
07d43f9
refactoring: bypass for same bug in second place
Olezha Dec 10, 2019
bc7ab73
disable snakeyaml implicit resolvers
Olezha Dec 10, 2019
10d6ea2
kludge related to bc7ab73
Olezha Dec 12, 2019
afee805
clean
Olezha Dec 12, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions build.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
<!-- ant build file for Freenet -->

<project name="plugins-Library" default="dist" basedir=".">
<property name="freenet-cvs-snapshot.location" location="../fred/dist/freenet.jar"/>
<property name="freenet-cvs-snapshot.location" location="../fred/build/libs/freenet.jar"/>
<property name="freenet-ext.location" location="../fred/lib/freenet/freenet-ext.jar"/>
<property name="source-version" value="1.6"/>
<property name="target-version" value="1.6"/>
<property name="source-version" value="1.8"/>
<property name="target-version" value="1.8"/>
<property name="build" location="build/"/>
<property name="build-test" location="build-test/"/>
<property name="run-test" location="run-test/"/>
Expand Down Expand Up @@ -64,14 +64,14 @@
<delete dir="${tmp}"/>
</target>

<property name="SnakeYAML.mirror" value="https://downloads.freenetproject.org/snakeyaml-1.5.jar" />
<property name="SnakeYAML.md5" value="1788db5d529a981b090632b8e49b8cd2" />
<property name="SnakeYAML.sha" value="a3654035d0b68db775034afa5425a62250de8184" />
<property name="SnakeYAML.mirror" value="https://repo1.maven.org/maven2/org/yaml/snakeyaml/1.25/snakeyaml-1.25.jar" />
<property name="SnakeYAML.md5" value="6f7d5b8f596047aae07a3bf6f23a0bf2" />
<property name="SnakeYAML.sha" value="8b6e01ef661d8378ae6dd7b511a7f2a33fae1421" />
<available property="SnakeYAML.exist" classname="org.yaml.snakeyaml.Yaml" classpathref="lib.path"/>
<target name="SnakeYAML-get" unless="SnakeYAML.exist">
<get verbose="true" src="${SnakeYAML.mirror}" dest="${tmp}/snakeyaml-1.5.jar" />
<checksum file="${tmp}/snakeyaml-1.5.jar" algorithm="MD5" property="${SnakeYAML.md5}" verifyProperty="SnakeYAMLMD5ok" />
<checksum file="${tmp}/snakeyaml-1.5.jar" algorithm="SHA" property="${SnakeYAML.sha}" verifyProperty="SnakeYAMLSHAok" />
<get verbose="true" src="${SnakeYAML.mirror}" dest="${tmp}/snakeyaml-1.25.jar" />
<checksum file="${tmp}/snakeyaml-1.25.jar" algorithm="MD5" property="${SnakeYAML.md5}" verifyProperty="SnakeYAMLMD5ok" />
<checksum file="${tmp}/snakeyaml-1.25.jar" algorithm="SHA" property="${SnakeYAML.sha}" verifyProperty="SnakeYAMLSHAok" />
<fail message="SnakeYAML.zip checksum mismatch">
<condition>
<or>
Expand All @@ -80,7 +80,7 @@
</or>
</condition>
</fail>
<move file="${tmp}/snakeyaml-1.5.jar" todir="lib/" />
<move file="${tmp}/snakeyaml-1.25.jar" todir="lib/" />
</target>

<!--<property name="KeyExplorer.mirror" value="http://cloud.github.com/downloads/freenet/plugin-Library-staging/KeyExplorer-dacfafecbc82aecdeffa56bef4a047a7f6c7f08d.jar" />
Expand Down Expand Up @@ -131,7 +131,7 @@
</javac>

<!-- Force compile of Version.java in case compile of ${src} didn't trigger it -->
<javac srcdir="${build}" destdir="${build}" debug="on" optimize="on" source="1.5" target="1.5">
<javac srcdir="${build}" destdir="${build}" debug="on" optimize="on" source="1.8" target="1.8">
<classpath>
<path refid="lib.path"/>
</classpath>
Expand All @@ -146,7 +146,7 @@
</target>

<target name="compile-tester" depends="compile" if="tester">
<javac srcdir="test/" destdir="${build}" debug="on" optimize="on" source="1.5" target="1.5">
<javac srcdir="test/" destdir="${build}" debug="on" optimize="on" source="1.8" target="1.8">
<classpath>
<path refid="lib.path"/>
<!-- Uses some code that uses junit -->
Expand All @@ -173,7 +173,7 @@

<!-- ================================================== -->
<target name="unit-build" depends="compile" if="junit.present" unless="skip_tests">
<javac srcdir="test/" destdir="${build-test}" debug="on" optimize="on" source="1.5">
<javac srcdir="test/" destdir="${build-test}" debug="on" optimize="on" source="1.8">
<classpath>
<path refid="lib.path"/>
<pathelement path="${build}"/>
Expand Down
2 changes: 1 addition & 1 deletion src/plugins/Library/SpiderIndexUploader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,7 @@ private Closure<Entry<String, SkeletonBTreeSet<TermEntry>>, TaskAbortException>
/*@Override**/ public void invoke(Map.Entry<String, SkeletonBTreeSet<TermEntry>> entry) throws TaskAbortException {
String key = entry.getKey();
SkeletonBTreeSet<TermEntry> tree = entry.getValue();
if(logMINOR) Logger.minor(this, "Processing: "+key+" : "+tree);
Logger.minor(this, "Processing: " + key + " : " + (tree != null ? tree : "new"));
if(tree != null)
Logger.debug(this, "Merging data (on disk) in term "+key);
else
Expand Down
59 changes: 38 additions & 21 deletions src/plugins/Library/index/ProtoIndexSerialiser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
import plugins.Library.util.SkeletonBTreeSet;
import plugins.Library.util.exec.SimpleProgress;
import plugins.Library.util.exec.TaskAbortException;
import plugins.Library.io.serial.Serialiser.*;
import plugins.Library.io.serial.LiveArchiver;
import plugins.Library.io.serial.Serialiser;
import plugins.Library.io.serial.Translator;
Expand All @@ -20,16 +19,9 @@

import freenet.keys.FreenetURI;

import java.util.Collection;
import java.util.Set;
import java.util.Map;
import java.util.SortedMap;
import java.util.SortedSet;
import java.util.LinkedHashMap;
import java.util.HashMap;
import java.util.TreeSet;
import java.util.TreeMap;
import java.util.Date;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.*;
import java.io.File;

/**
Expand Down Expand Up @@ -187,27 +179,52 @@ public IndexTranslator(LiveArchiver<Map<String, Object>, SimpleProgress> subsrl)
}

/*@Override**/ public ProtoIndex rev(Map<String, Object> map) throws DataFormatException {
long magic = (Long)map.get("serialVersionUID");
Object serialVersionUID = map.get("serialVersionUID");
long magic;
if (serialVersionUID instanceof String) { // FIXME
magic = Long.parseLong((String) map.get("serialVersionUID"));
} else {
magic = (Long) serialVersionUID;
}

if (magic == ProtoIndex.serialVersionUID) {
try {
// FIXME yet more hacks related to the lack of proper asynchronous FreenetArchiver...
ProtoIndexComponentSerialiser cmpsrl = ProtoIndexComponentSerialiser.get((Integer)map.get("serialFormatUID"), subsrl);
Object serialFormatUIDObj = map.get("serialFormatUID");
int serialFormatUID;
if (serialFormatUIDObj instanceof String) { // FIXME
Copy link

@Bombe Bombe Jul 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does everything in this map potentially exist as String? Where does this map come from?

Copy link
Contributor

@Olezha Olezha Aug 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could write this in the situation when I met a String there during debugging. Dirty. In my opinion it starts with "snakeyaml implicit resolver", which I turned off as a result.

serialFormatUID = Integer.parseInt((String) map.get("serialFormatUID"));
} else {
serialFormatUID = (Integer) serialFormatUIDObj;
}
ProtoIndexComponentSerialiser cmpsrl = ProtoIndexComponentSerialiser.get(serialFormatUID, subsrl);
FreenetURI reqID = (FreenetURI)map.get("reqID");
String name = (String)map.get("name");
String ownerName = (String)map.get("ownerName");
String ownerEmail = (String)map.get("ownerEmail");
// FIXME yaml idiocy??? It seems to give a Long if the number is big enough to need one, and an Integer otherwise.
Object totalPagesObj = map.get("totalPages");
long totalPages;
Object o = map.get("totalPages");
if(o instanceof Long)
totalPages = (Long)o;
else // Integer
totalPages = (Integer)o;
Date modified = (Date)map.get("modified");
if (totalPagesObj instanceof String) { // FIXME
totalPages = Long.parseLong((String) totalPagesObj);
} else if (totalPagesObj instanceof Long) { // FIXME yaml??? It seems to give a Long if the number
Copy link

@Bombe Bombe Jul 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use Number and it’s longValue() method.

totalPages = (Long) totalPagesObj; // is big enough to need one, and an Integer otherwise.
} else {
totalPages = (Integer) totalPagesObj;
}
Object modifiedObj = map.get("modified");
Date modified;
if (modifiedObj instanceof String) { // FIXME
try {
modified = new SimpleDateFormat("yyyy-MM-dd").parse((String) modifiedObj);
} catch (ParseException ignored) {
modified = null;
}
} else {
modified = (Date) modifiedObj;
}
Map<String, Object> extra = (Map<String, Object>)map.get("extra");
SkeletonBTreeMap<URIKey, SkeletonBTreeMap<FreenetURI, URIEntry>> utab = utrans.rev((Map<String, Object>)map.get("utab"));
SkeletonBTreeMap<String, SkeletonBTreeSet<TermEntry>> ttab = ttrans.rev((Map<String, Object>)map.get("ttab"));
SkeletonBTreeMap<String, SkeletonBTreeSet<TermEntry>> ttab = ttrans.rev((Map<String, Object>) map.get("ttab"));

return cmpsrl.setSerialiserFor(new ProtoIndex(reqID, name, ownerName, ownerEmail, totalPages, modified, extra, utab, ttab));

Expand Down
3 changes: 2 additions & 1 deletion src/plugins/Library/index/TermEntry.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ public enum EntryType {

public TermEntry(String s, float r) {
if (s == null) {
throw new IllegalArgumentException("can't have a null subject!");
// throw new IllegalArgumentException("can't have a null subject!");
s = "null"; // FIXME
Copy link

@Bombe Bombe Jul 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this FIXME in combination with the removed exception does not inspire confidence. How is using "null" as subject better than getting an exception and fixing the cause of that?

Copy link
Collaborator Author

@desyncr desyncr Jul 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably related to https://github.com/freenet/plugin-Library/pull/16/files#diff-aa254c0343e62a1fcd16f7077a106f09R302. Not sure if that gives away a bit more of context.

Copy link
Contributor

@Olezha Olezha Aug 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can remember this situation, this is about the fact that the subject can contain the human word "null", but then it turns into null (smart library mode)

Copy link

@Bombe Bombe Aug 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like something that should be fixed in a different place… but as with the rest of this PR I’m totally not qualified to talk about any of it because I have no clue of all the contexts in question. :)

}
if (r < 0/* || r > 1*/) { // FIXME: I don't see how our relevance algorithm can be guaranteed to produce relevance <1.
throw new IllegalArgumentException("Relevance must be in the half-closed interval (0,1]. Supplied: " + r);
Expand Down
3 changes: 3 additions & 0 deletions src/plugins/Library/io/ObjectBlueprint.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,9 @@ public T objectFromMap(Map<?, ?> map) throws InstantiationException, IllegalAcce
String property = en.getKey();
Class<?> type = en.getValue();
Object value = map.get(property);
if (value != null && value.equals("null")) { // FIXME: case when type Map and value String "null"
value = null;
}
try {
if (type.isPrimitive()) {
value = boxCast(type, value);
Expand Down
Loading