diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..6f1682c --- /dev/null +++ b/.gitignore @@ -0,0 +1,9 @@ +.DS_Store +.idea +packages +.project +local_pubspec_yaml +pubspec.lock +.idea +*.swp +build diff --git a/AUTHORS b/AUTHORS new file mode 100644 index 0000000..1218bcd --- /dev/null +++ b/AUTHORS @@ -0,0 +1,7 @@ +Paul Brauner +Rafael Brandão +Jozef Brandys +Roman Hudec +Marian Hornak +Tomas Kulich +Matus Fedak diff --git a/LICENSE b/LICENSE index cb0b567..64952f4 100644 --- a/LICENSE +++ b/LICENSE @@ -1,3 +1,4 @@ +Copyright 2014, VacuumLabs. Copyright 2012, Google Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: diff --git a/README.md b/README.md index c672a97..7d66817 100644 --- a/README.md +++ b/README.md @@ -1,50 +1,76 @@ # Efficient Persistent Data Structures -[![Build Status](https://drone.io/github.com/polux/persistent/status.png)](https://drone.io/github.com/polux/persistent/latest) +[![Build Status](https://drone.io/github.com/vacuumlabs/persistent/status.png)](https://drone.io/github.com/vacuumlabs/persistent/latest) -Mostly efficient persistent maps and sets for now. Also option types. +Check out [changes in 2.0 version!] (changes_2_0.md) -"Persistent" means immutable here, not "saved on disk". +Learn how you can use [transients] (transients.md) -```dart -import 'package:persistent/persistent.dart'; +Want to understand the code? Want to contribute? See [technical overview] (technical.md) -main() { - final emptyMap = new PersistentMap(); - final m1 = emptyMap.insert('a', 1).insert('b', 2); - final m2 = new PersistentMap.fromMap({'a': 3, 'c': 4}); + - print(m1); // {a: 1, b: 2} - print(m2); // {c: 4, a: 3} - print(m1.lookup('a')); // Option.some(1) - print(m1.lookup('c')); // Option.none() +## What are persistent data structures +*Persistent* data structure is an immutable structure; the main difference with standard data structures is how you 'write' to them: instead of mutating +the old structure, you create the new, independent, (slightly) modified copy of it. Typical examples of commonly used Persistent structures are String (in Java, Javascript, Python, Ruby) or Python's Tuple or Java's BigDecimal. [(Not only)](http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey) we believe such concept could be beneficial also for other data structures such as Maps, Lists/Vectors, Sets. - final m3 = m1.delete('a'); - print(m1); // {a: 1, b: 2} - print(m3); // {b: 2} + var couple = new PMap.fromMap({'father': 'Homer', 'mother': 'Marge'}); + // do not (and can not) modify couple anymore + var withChild = couple.assoc('boy', 'Bart'); + print(couple); // {mother: Marge, father: Homer} + print(withChild); // {boy: Bart, mother: Marge, father: Homer} - final m4 = m1.union(m2, (n,m) => n + m); - print(m4); // {c: 4, a: 4, b: 2} +## Got it. And it is cool because...? - final m5 = m1.mapValues((n) => n + 1); - print(m5); // {a: 2, b: 3} +### It disallows unwanted side effects +You know the story. Late in the evening, exhausted and frustrated you find out that some guy that implemented - final m6 = m1.adjust('a', (n) => n + 1); - print(m6); // {a: 2, b: 2} -} -``` + int ComputeResponseLength(Map responseMap) -## Try it! +got a 'great' idea, that instead of just computing response length he also mutates responseMap in some tricky way (say, he does some kind of sanitization of responseMap). Even if this was mentioned in the documentation and even if the methods name was different: this is a spaghetti code. -``` -git clone https://github.com/polux/persistent.git -cd persistent -pub install -dart example/map_example.dart -dart example/set_example.dart -dart test/map_bench.dart -``` +### Equality and hashCode done right +Finally, they work as you'd expect. How cool is this: -## More + // deeply persist the structure of Maps and Lists + PMap a = persist({[1,2]: 'tower', [1,3]: 'water'}); + PMap b = persist({[1,2]: 'tower', [1,3]: 'water'}); + assert(a==b); + // kids, don't try this with standard List, it ain't going to work + print(a[persist([1, 2])]); // prints 'tower' + +### Instant 'deep copy' +Just copy/pass the reference. It's like doing deep copy in O(1). + +### Caching made easier +Caching can speed things up significantly. But how do you cache results of a function + + List findSuspiciousEntries(List entries) + +One possible workaround would be to JSONize entries to string and use such string as a hashing key. However, it's much more elegant, safe (what about ordering of keys within maps?), performant and memory-wise with Persistent structures. Also, until you restrict to well-behaving functions, there's no need to invalidate cache; you can cache anything, for as long as you want. + +### Simplicity matters +Fast copying or equality done right are nice features, but this is not the only selling point here. Having different ways how to copy (shallow, deep) objects or how to compare them (== vs. equals in Java) introduces new complexity. Even if you get used to it, it still takes some part of your mental capabilities and can lead to errors. + +### Structure sharing + PMap map1 = persist({'a': 'something', 'b': bigMap}); + PMap map2 = a.assoc('a', 'something completely different'); +Suppose you are interested, whether map1['b'] == map2['b']. Thanks to structure sharing, this is O(1) operation, which means it is amazingly fast - no need to traverse through the bigMap. Although it may sound unimportant at the first glance, it is what really enables fast caching of complex functions. Also, this is the reason, why [Om](https://github.com/swannodette/om/) framework is MUCH faster than Facebooks [React](http://facebook.github.io/react/). + +## And what is the prize for this all +In the 2.0 release, we optimized memory consumption such that the only penalty for using Persistent +comes at lower speed. Although structure sharing makes the whole thing much more effective than naive +copy-it-all approach, Persistents are still slower than their mutable counterparts (note however, that on +the other hand, some operations runs significantly faster, so its hard to say something conclusive +here). Following numbers illustrate, how much slow are Persistent data structures when benchmarking either on DartVM +or Dart2JS on Node (the numbers are quite independent of the structure size): + +* DartVM read speed: 2 +* DartVM write speed: 12 (5 by using Transients) +* Dart2JS read speed: 3 +* Dart2JS write speed: 14 (6 by using Transients) + +Although the factors are quite big, the whole operation is still very fast and it probably won't be THE bottleneck which would slow down your app. -See [ImplementationDetails](https://github.com/polux/persistent/wiki/ImplementationDetails) and the [generated API documentation](http://polux.github.io/persistent/continuous/persistent/PersistentMap.html) for more information. diff --git a/benchmark/.gitignore b/benchmark/.gitignore new file mode 100644 index 0000000..c28777f --- /dev/null +++ b/benchmark/.gitignore @@ -0,0 +1,4 @@ +*.js +*.js.map +*.js.deps +temp.out diff --git a/benchmark/map_bench.dart b/benchmark/map_bench.dart deleted file mode 100644 index 5ecfa00..0000000 --- a/benchmark/map_bench.dart +++ /dev/null @@ -1,22 +0,0 @@ -// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code -// is governed by a BSD-style license that can be found in the LICENSE file. - -// Author: Paul Brauner (polux@google.com) - -library map_bench; - -import 'package:persistent/persistent.dart'; - -import 'dart:math'; - -part 'src/benchmark.dart'; -part 'src/simple_map_1.dart'; -part 'src/simple_map_2.dart'; - -void main() { - Benchmark.warmup(); - for (int n = 0; n < 20000; n += 100) { - Benchmark benchmark = new Benchmark(n); - print("$n: ${benchmark.bench()}"); - } -} diff --git a/benchmark/map_bench_wordcount.csv b/benchmark/map_bench_wordcount.csv deleted file mode 100644 index dcedf2d..0000000 --- a/benchmark/map_bench_wordcount.csv +++ /dev/null @@ -1,23 +0,0 @@ -Dart VM version: 0.5.0.1_r21823 (Mon Apr 22 14:02:11 2013), 535c1ae834b3bb23f004a712f36fdf8681d7d5db, Wed Apr 24 22:48:18 UTC 2013, 1178.0 -Dart VM version: 0.5.0.1_r21823 (Mon Apr 22 14:02:11 2013), 30cf32f6e403b60a8734b2d8780f39bcdee339e3, Thu Apr 25 17:40:55 UTC 2013, 1186.4 -Dart VM version: 0.5.1.0_r22072 (Fri Apr 26 12:54:33 2013), 32df58d26606334e00338c947c4f00fbdfc277ae, Mon Apr 29 21:16:52 UTC 2013, 883.4 -Dart VM version: 0.5.5.0_r22416 (Mon May 6 10:31:00 2013), c4c634e4d987c1461aa80e3d8627623ec04165f6, Tue May 7 21:58:28 UTC 2013, 663.5 -Dart VM version: 0.5.7.2_r22611 (Fri May 10 22:19:57 2013) on "linux_x64", 82ef833adcb9ff1af44ca3d02431cec36a605451, Mon May 13 23:00:06 UTC 2013, 2397.7 -Dart VM version: 0.5.9.0_r22879 (Sat May 18 21:32:26 2013) on "linux_x64", 18e8301013b444cc0a0ba89605dd784a6baf0546, Tue May 21 00:35:15 UTC 2013, 501.3 -Dart VM version: 0.5.11.1_r23200 (Mon May 27 00:02:29 2013) on "linux_x64", c08c2f50683e6ad8647ec2572bbb99a1d368a819, Wed Jun 5 06:18:28 UTC 2013, 581.8 -Dart VM version: 0.5.20.2_r24160 (Tue Jun 18 20:43:32 2013) on "linux_x64", 4004a85d1e6e9a9b16a39ea7d81b6cac7d80dc2f, Wed Jun 19 19:09:31 UTC 2013, 513.6 -Dart VM version: 0.6.3.3_r24898 (Thu Jul 11 07:47:12 2013) on "linux_x64", b1323f8a6bd330f89b48ca18e7821aab898a58c3, Tue Jul 16 00:01:25 UTC 2013, 522.8 -Dart VM version: 0.6.5.0_r25017 (Mon Jul 15 15:01:03 2013) on "linux_x64", 248ae0108cce466418185170214125b9ce0768bd, Wed Jul 17 22:35:09 UTC 2013, 528.6 -Dart VM version: 0.6.9.2_r25388 (Tue Jul 23 20:28:04 2013) on "linux_x64", 248ae0108cce466418185170214125b9ce0768bd, Fri Jul 26 20:37:00 UTC 2013, 506.0 -Dart VM version: 0.6.13.0_r25630 (Tue Jul 30 14:33:28 2013) on "linux_x64", 8c941a35b8168300626389780bacee56c6842552, Wed Jul 31 17:12:06 UTC 2013, 553.8 -Dart VM version: 0.6.15.3_r25822 (Tue Aug 6 13:39:55 2013) on "linux_x64", 196bed96bf2a209249aa945658e51567febe5006, Sun Aug 11 20:50:07 UTC 2013, 518.6 -Dart VM version: 0.7.3.1_r27487 (Fri Sep 13 13:27:39 2013) on "linux_x64", e1fbb8e11d30a9d7f7788d8846755aac296a0af4, Tue Sep 17 22:38:58 UTC 2013, 541.5 -Dart VM version: 0.7.5.3_r27776 (Mon Sep 23 14:31:35 2013) on "linux_x64", 800e9cc2bc700d9d87533772e8b537ae0fd42e46, Tue Sep 24 19:20:20 UTC 2013, 556.3 -Dart VM version: 0.7.6.4_r28108 (Tue Oct 1 14:32:31 2013) on "linux_x64", 5c80ab2f0f72d2607b41a2d08c80fe680449053b, Thu Oct 3 00:54:55 UTC 2013, 519.6 -Dart VM version: 0.8.5.1_r28990 (Tue Oct 22 04:50:58 2013) on "linux_x64", ffb7eee674f3be752956e8d99a9079ab4d3d8e3c, Tue Oct 22 19:41:31 UTC 2013, 559.7 -Dart VM version: 0.8.7.0_r29341 (Mon Oct 28 02:00:50 2013) on "linux_x64", 147768fc8239ad23d1074b5f4029d7dab0f2113d, Mon Oct 28 22:13:49 UTC 2013, 510.3 -Dart VM version: 0.8.10.8_r30039 (Thu Nov 7 03:03:33 2013) on "linux_x64", bd99a6c075b968fee23c541eee631e6396fbc262, Thu Nov 7 22:28:22 UTC 2013, 533.3 -Dart VM version: 1.1.0-dev.5.0 (Fri Dec 20 05:55:37 2013) on "linux_x64", 4cdf453c59369653d3e9432632ef4420e732fb36, Sat Jan 4 15:40:45 UTC 2014, 432.6 -Dart VM version: 1.2.0-dev.4.0 (Fri Feb 7 10:21:55 2014) on "linux_x64", 41c8cd12714847609f1647a4df5c20131cde759e, Tue Feb 11 22:37:52 UTC 2014, 463.3 -Dart VM version: 1.5.0-dev.2.0 (Mon May 26 07:29:28 2014) on "linux_x64", d3a3d1b4917de54dc1ddbe2f61169b35f21f85fb, Sun Jun 1 16:40:46 UTC 2014, 359.7 -Dart VM version: 1.5.0-dev.4.7 (Wed Jun 11 01:57:57 2014) on "linux_x64", dc6fe1c592a83e15d798a37f77013cafcb46c500, Thu Jun 12 09:00:32 UTC 2014, 345.6 diff --git a/benchmark/map_bench_wordcount.dart b/benchmark/map_bench_wordcount.dart deleted file mode 100644 index 7bb12c2..0000000 --- a/benchmark/map_bench_wordcount.dart +++ /dev/null @@ -1,42 +0,0 @@ -import 'dart:async'; -import 'package:http/http.dart' as http; -import 'package:persistent/persistent.dart'; - -add(n, m) => n + m; -insertWord(map, word) => map.insert(word, 1, add); -count(words) => words.fold(new PersistentMap(), insertWord); - -final AUSTEEN = 'http://www.gutenberg.org/files/1342/1342.txt'; -final DOYLE = 'http://www.gutenberg.org/files/1661/1661.txt'; - -Future> getWords(String url) { - return http.get(url).then((response) => response.body.split(' ')); -} - -void run(List doyle, List austeen) { - final map1 = count(austeen); - final map2 = count(doyle); - final result = map1.union(map2, add); - if (result.length != 36028) { - throw new StateError("something's wrong"); - } -} - -void bench(List doyle, List austeen) { - // warmup - run(doyle, austeen); - - final watch = new Stopwatch(); - watch.start(); - for (int i = 0; i < 10; i++) { - run(doyle, austeen); - } - watch.stop(); - print(watch.elapsedMilliseconds / 10); -} - -main() { - Future - .wait([getWords(AUSTEEN), getWords(DOYLE)]) - .then((result) => bench(result[0], result[1])); -} diff --git a/benchmark/map_memory/makefile b/benchmark/map_memory/makefile new file mode 100644 index 0000000..63de5bf --- /dev/null +++ b/benchmark/map_memory/makefile @@ -0,0 +1,62 @@ +template_size = 1000 + +define RUNNER +function dartMainRunner(main, args){ + main(process.argv.slice(2)); +} +endef + +export RUNNER + +define INTRO +- - - - - - - - +Using (flat) map of size ${template_size}. Benchmark internally counts number of objects that fit +into 1GB heap space. Output is in bytes per one key->value storage place. Instances of keys and +values are shared between multiple copies of the structure we create, therefore their size do not +affect the result (much). +- - - - - - - - +endef + +export INTRO + + +all: clean run + + +build/map_memory.js: map_memory.dart + mkdir -p build + dart2js -o build/tmp.js map_memory.dart + cat ../preamble/d8.js >> build/map_memory.js + cat build/tmp.js >> build/map_memory.js + echo "$$RUNNER" >> build/map_memory.js + +clean: + rm -rf build + + +run: build/map_memory.js + + @echo "$$INTRO" + +# @echo -n "DartVM persistent: " +# @dart --old_gen_heap_size=1024 map_memory.dart ${template_size} persistent > build/temp.out 2>/dev/null +# @tail -1 build/temp.out +# +# @echo -n "DartVM map: " +# @dart --old_gen_heap_size=1024 map_memory.dart ${template_size} map > build/temp.out 2>/dev/null +# @tail -1 build/temp.out + + @echo -n "NodeJS persistent: " + @ (! node --max-old-space-size=1024 build/map_memory.js ${template_size} persistent > build/temp.out) 2>/dev/null + @tail -1 build/temp.out + + @echo -n "NodeJS map: " + @ (! node --max-old-space-size=1024 build/map_memory.js ${template_size} map > build/temp.out) 2>/dev/null + @tail -1 build/temp.out + + @echo -n "NodeJS hashmap: " + @ (! node --max-old-space-size=1024 build/map_memory.js ${template_size} hashmap > build/temp.out) 2>/dev/null + @tail -1 build/temp.out + + rm -f build/temp.out + diff --git a/benchmark/map_memory/map_memory.dart b/benchmark/map_memory/map_memory.dart new file mode 100644 index 0000000..58f9c74 --- /dev/null +++ b/benchmark/map_memory/map_memory.dart @@ -0,0 +1,54 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +library map_memory; + +import 'package:vacuum_persistent/persistent.dart'; +import 'dart:collection'; + +Map template = {}; + +// here we will store the data to prevent it from garbage collection +List data = []; + +var creators = { + + "persistent": () => new PMap.fromMap(template), + + "transient": (){ + var res = new TMap(); + template.forEach((k, v) => res.doAssoc(k, v)); + return res; + }, + + "map": () => new Map.from(template), + "hashmap": () => new HashMap.from(template) +}; + +void run(int template_size, String mode) { + + for (int i = 0; i < template_size; i++) { + template["$i".padLeft(8)] = "$i".padRight(8); + } + + int allocated = 0; + for(bool go = true; go; allocated++){ + try{ + go = false; + var a = creators[mode](); + data.add(a); + go = true; + print(1073741824.0 / allocated / template_size); + } catch(e) { + data = null; + } + } +} + +main(List args){ + + run(int.parse(args[0]), args[1]); +} diff --git a/benchmark/map_speed/benchmarks.dart b/benchmark/map_speed/benchmarks.dart new file mode 100644 index 0000000..9272fff --- /dev/null +++ b/benchmark/map_speed/benchmarks.dart @@ -0,0 +1,72 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of map_bench; + +class WriteBenchmark extends BenchmarkBase{ + + final Map sample; + BenchmarkInterface object; + final dynamic factory; + + void setup(){ + object = factory(); + } + + WriteBenchmark(this.sample, this.factory):super('Writing'); + + void run(){ + for (var size in this.sample.keys) { + for (var j=0; j sample; + Map> objects = new Map(); + final dynamic factory; + + ReadBenchmark(this.sample, this.factory):super('Reading'); + + void setup(){ + this.sample.forEach((size, count){ + objects[size] = []; + for (int j=0; j= 0; i--) { + object.get(i*i); + } + } + } + } +} + + diff --git a/benchmark/map_speed/interface.dart b/benchmark/map_speed/interface.dart new file mode 100644 index 0000000..c42d30b --- /dev/null +++ b/benchmark/map_speed/interface.dart @@ -0,0 +1,35 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of map_bench; + +abstract class BenchmarkInterface{ + + void assoc(K key, V value); + void get(K key); + void delete(K key); + + void save(); + void restore(); +} + + +abstract class EncapsulatingInterface + extends BenchmarkInterface{ + + T object = null; + T object_copy = null; + + T _copy(); + + save(){ + object_copy = _copy(); + } + + restore(){ + return object = object_copy; + } +} diff --git a/benchmark/map_speed/interface_impl.dart b/benchmark/map_speed/interface_impl.dart new file mode 100644 index 0000000..a37e4b1 --- /dev/null +++ b/benchmark/map_speed/interface_impl.dart @@ -0,0 +1,127 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of map_bench; + +class PersistentMapInterface + extends EncapsulatingInterface>{ + + PersistentMapInterface(){ + this.object = new PMap(); + } + + assoc(K key, V value) => + object = object.assoc(key, value); + + get(K key) => object.get(key, null); + + delete(K key) => object = object.delete(key); + + _copy() => object; +} + + +class TransientMapInterface + extends EncapsulatingInterface>{ + + TransientMapInterface(){ + this.object = new PMap().asTransient(); + } + + assoc(K key, V value) => + object.doAssoc(key, value); + + get(K key) => object.get(key, null); + + delete(K key) => object.doDelete(key); + + _copy(){ + var copy = object.asPersistent(); + object = copy.asTransient(); + return copy.asTransient(); + } +} + + +class StandardMapInterface + extends EncapsulatingInterface>{ + + StandardMapInterface(){ + object = new Map(); + } + + assoc(K key, V value) => + object[key] = value; + + get(K key) => object[key]; + + delete(K key) => object.remove(key); + + _copy() => new Map.from(object); +} + + +class CopyMapInterface + extends EncapsulatingInterface>{ + + create() => object = new Map(); + + assoc(K key, V value){ + object = new Map.from(object); + object[key] = value; + } + + get(K key) => object[key]; + + delete(K key){ + object = new Map.from(object); + object.remove(key); + } + + _copy() => object; +} + + +class LinkedListInterface + extends EncapsulatingInterface>>{ + + create() => object = new Nil>(); + + assoc(K key, V value){ + LinkedListBuilder> builder = + new LinkedListBuilder>(); + LinkedList> it = object; + while (it.isCons) { + Cons> cons = it.asCons; + Pair elem = cons.elem; + if (elem.fst == key) { + builder.add(new Pair(key, value)); + return builder.build(cons.tail); + } + builder.add(elem); + it = cons.tail; + } + builder.add(new Pair(key, value)); + object = builder.build(); + } + + get(K key){ + LinkedList> it = object; + while (it.isCons) { + Cons> cons = it.asCons; + Pair elem = cons.elem; + if (elem.fst == key) return; + it = cons.tail; + } + } + + delete(K key){ + object = object.strictWhere((p) => p.fst != key); + } + + _copy() => object; + } + diff --git a/benchmark/map_speed/makefile b/benchmark/map_speed/makefile new file mode 100644 index 0000000..271ea89 --- /dev/null +++ b/benchmark/map_speed/makefile @@ -0,0 +1,26 @@ +all: clean run + + +build/map_speed.js: map_speed.dart + mkdir -p build + dart2js -o build/tmp.js map_speed.dart + cat ../preamble/d8.js >> build/map_speed.js + cat build/tmp.js >> build/map_speed.js + +clean: + rm -rf build + + +run: build/map_speed.js + + @echo "Performing several insert, lookup and delete operations on flat maps." + @echo "The number of operations depends on the map size." + + @echo "--------" + @echo "DartVM:" + @dart map_speed.dart + + @echo "--------" + @echo "NodeJS:" + @node build/map_speed.js + diff --git a/benchmark/map_speed/map_speed.dart b/benchmark/map_speed/map_speed.dart new file mode 100644 index 0000000..5241d9e --- /dev/null +++ b/benchmark/map_speed/map_speed.dart @@ -0,0 +1,72 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +library map_bench; + +import 'package:vacuum_persistent/persistent.dart'; +import 'package:benchmark_harness/benchmark_harness.dart'; +import 'dart:math'; + +part 'benchmarks.dart'; +part 'interface.dart'; +part 'interface_impl.dart'; + +Map interfaces = { + "PersistentMap": () => new PersistentMapInterface(), + "TransientMap": () => new TransientMapInterface(), + "Map": () => new StandardMapInterface(), +}; + +int times = 4; + +void main() { + + var config = [ + {'name': 'Write', + 'creator': ((sample, factory) => (new WriteBenchmark(sample, factory))), +// 'sizes': [{10000: 1}, ], + 'sizes': [{500:60, 1000: 30, 1500: 20, 3000: 10}], + }, + { + 'name': 'Read', + 'creator': ((sample, factory) => (new ReadBenchmark(sample, factory))), +// 'sizes': [{10000: 1}, ], + 'sizes': [{500:60, 1000: 30, 1500: 20, 3000: 10}], + } + ]; + var result = {}; + config.forEach((conf){ + String mode = conf['name']; + var creator = conf['creator']; + for (Map sample in conf['sizes']) { + var res = {}; + var dev = {}; + interfaces.forEach((k,v){ + res[k] = 0; + dev[k] = 0; + }); + for (int i=0; i temp.out) 2>/dev/null + @tail -1 temp.out + + @echo -n "NodeJS vanilla JS map: " + @ (! node --max-old-space-size=1024 memory.js ${template_size} map > temp.out) 2>/dev/null + @tail -1 temp.out + + @echo -n "NodeJS Mori persistent vector: " + @ (! node --max-old-space-size=1024 memory.js ${template_size} persistent_vector > temp.out) 2>/dev/null + @tail -1 temp.out + + @echo -n "NodeJS vanilla JS list: " + @ (! node --max-old-space-size=1024 memory.js ${template_size} list > temp.out) 2>/dev/null + @tail -1 temp.out + + rm -f temp.out + diff --git a/benchmark/mori/memory.js b/benchmark/mori/memory.js new file mode 100644 index 0000000..5816af8 --- /dev/null +++ b/benchmark/mori/memory.js @@ -0,0 +1,62 @@ +var m = require('mori'); + +var template_map = {}; +var template_vector = []; + +// here we will store the data to prevent it from garbage collection +data = []; + +// JSON.stringify & parse is not good way of cloning since it creates copies of the values; we write +// custom (shallow) clone functions instead + +function clone_list(template){ + var res = []; + for (var i=0; i compiled.js + +run: compiled.js + node compiled.js + +clean: + rm -f compiled.js + + diff --git a/benchmark/mori_speed/mori_speed.js b/benchmark/mori_speed/mori_speed.js new file mode 100644 index 0000000..4e00750 --- /dev/null +++ b/benchmark/mori_speed/mori_speed.js @@ -0,0 +1,111 @@ +var Benchmark = require('benchmark'); +var m = require('mori'); + +var samples = [ + {"500":60, "1000":30, "1500":20, "3000":10} +] + +var times = 10; + +// Read Map + +var read_map_suite = new Benchmark.Suite; + +for (var sample_id = 0; sample_id < samples.length; sample_id++){ + + var sample = samples[sample_id]; + var objects = {} + + for(var size_name in sample){ + var size = Number(size_name) + var count = sample[size_name] + + var prototype = [] + for (var i = 0; i < size; i++){ + prototype.push(String(i*i)) + prototype.push("foo"); + } + + subobjects = []; + for (var i = 0; i < count; i++){ + subobjects.push(m.hashMap.apply(null, prototype)); + } + + objects[size_name] = subobjects; + } + + read_map_suite.add('Map Read on '+JSON.stringify(sample), function() { + for(var size_name in sample){ + var size = Number(size_name) + + objects[size_name].forEach(function(map){ + for (var i = size; i >= 0; i--) { + // somefn.f2, .f1, .f3, etc were added in mori 0.3 to speed-up method dispatching. + // .f2 means version of function with arity 2, etc + m.get.f2(map, i*i); + } + for (var i = 0; i <= size; i++) { + m.get.f2(map, i*i); + } + }); + + } + }, {"minSamples": times}); + +} + + + +// Write Map + +var write_map_suite = new Benchmark.Suite; + +for (var sample_id = 0; sample_id < samples.length; sample_id++){ + + var sample = samples[sample_id]; + + write_map_suite.add('Map Write on '+JSON.stringify(sample), function() { + for(var size_name in sample){ + var size = Number(size_name); + var count = sample[size_name]; + for (var j = 0; j < count; j++){ + var map = m.hashMap(); + vals = ["foo", "bar", "baz", "woo", "hoo", "goo", "wat"]; + for(var ival in vals){ + var val = vals[ival]; + for (var i = 0; i < size; i++) { + map = m.assoc(map, i*i, val); + } + } + for (var i = 0; i < size; i++) { + map = m.dissoc(map, i*i); + } + } + } + }, {"minSamples": times}); + +} + + + + +// Run + +var logger = function(event) { + var error = event.target.error, + time = (1/event.target.hz * 1000000); + stats = event.target.stats, + size = stats.sample.length, + result = event.target.name; + + if (error) { + result += ': ' + join(error); + } else { + result += ': ' + time.toFixed(0) + ' us +/-' + + stats.rme.toFixed(2) + '% (' + size + ' runs sampled)'; + } + console.log(result); +} + +read_map_suite.on('cycle', logger).run(); +write_map_suite.on('cycle', logger).run(); diff --git a/benchmark/mori_speed/package.json b/benchmark/mori_speed/package.json new file mode 100644 index 0000000..9b48929 --- /dev/null +++ b/benchmark/mori_speed/package.json @@ -0,0 +1,10 @@ +{ + "name": "mori_speed", + "main": "mori_speed.js", + "dependencies": { + "benchmark": "^1.0.0", + "closurecompiler": "^1.5.1", + "mori": "0.3.0" + }, + "private": true +} diff --git a/benchmark/preamble/README b/benchmark/preamble/README new file mode 100644 index 0000000..7b6d65e --- /dev/null +++ b/benchmark/preamble/README @@ -0,0 +1,2 @@ +The d8.js is polyfill some of the functionality that browsers provide. For more info: +http://dart.googlecode.com/svn/branches/1.8/dart/sdk/lib/_internal/compiler/js_lib/preambles/README diff --git a/benchmark/preamble/d8.js b/benchmark/preamble/d8.js new file mode 100644 index 0000000..aed0161 --- /dev/null +++ b/benchmark/preamble/d8.js @@ -0,0 +1,283 @@ +// Copyright (c) 2014, the Dart project authors. Please see the AUTHORS file +// for details. All rights reserved. Use of this source code is governed by a +// BSD-style license that can be found in the LICENSE file. + +// Javascript preamble, that lets the output of dart2js run on V8's d8 shell. + +// Node wraps files and provides them with a different `this`. The global +// `this` can be accessed through `global`. + +var self = this; +if (typeof global != "undefined") self = global; // Node.js. + +(function(self) { + // Using strict mode to avoid accidentally defining global variables. + "use strict"; // Should be first statement of this function. + + // Location (Uri.base) + + var workingDirectory; + // TODO(sgjesse): This does not work on Windows. + if (typeof os == "object" && "system" in os) { + // V8. + workingDirectory = os.system("pwd"); + var length = workingDirectory.length; + if (workingDirectory[length - 1] == '\n') { + workingDirectory = workingDirectory.substring(0, length - 1); + } + } else if (typeof process != "undefined" && + typeof process.cwd == "function") { + // Node.js. + workingDirectory = process.cwd(); + } + self.location = { href: "file://" + workingDirectory + "/" }; + + // Event loop. + + // Task queue as cyclic list queue. + var taskQueue = new Array(8); // Length is power of 2. + var head = 0; + var tail = 0; + var mask = taskQueue.length - 1; + function addTask(elem) { + taskQueue[head] = elem; + head = (head + 1) & mask; + if (head == tail) _growTaskQueue(); + } + function removeTask() { + if (head == tail) return; + var result = taskQueue[tail]; + taskQueue[tail] = undefined; + tail = (tail + 1) & mask; + return result; + } + function _growTaskQueue() { + // head == tail. + var length = taskQueue.length; + var split = head; + taskQueue.length = length * 2; + if (split * 2 < length) { // split < length / 2 + for (var i = 0; i < split; i++) { + taskQueue[length + i] = taskQueue[i]; + taskQueue[i] = undefined; + } + head += length; + } else { + for (var i = split; i < length; i++) { + taskQueue[length + i] = taskQueue[i]; + taskQueue[i] = undefined; + } + tail += length; + } + mask = taskQueue.length - 1; + } + + // Mapping from timer id to timer function. + // The timer id is written on the function as .$timerId. + // That field is cleared when the timer is cancelled, but it is not returned + // from the queue until its time comes. + var timerIds = {}; + var timerIdCounter = 1; // Counter used to assing ids. + + // Zero-timer queue as simple array queue using push/shift. + var zeroTimerQueue = []; + + function addTimer(f, ms) { + var id = timerIdCounter++; + f.$timerId = id; + timerIds[id] = f; + if (ms == 0) { + zeroTimerQueue.push(f); + } else { + addDelayedTimer(f, ms); + } + return id; + } + + function nextZeroTimer() { + while (zeroTimerQueue.length > 0) { + var action = zeroTimerQueue.shift(); + if (action.$timerId !== undefined) return action; + } + } + + function nextEvent() { + var action = removeTask(); + if (action) { + return action; + } + do { + action = nextZeroTimer(); + if (action) break; + var nextList = nextDelayedTimerQueue(); + if (!nextList) { + return; + } + var newTime = nextList.shift(); + advanceTimeTo(newTime); + zeroTimerQueue = nextList; + } while (true) + var id = action.$timerId; + clearTimerId(action, id); + return action; + } + + // Mocking time. + var timeOffset = 0; + var now = function() { + // Install the mock Date object only once. + // Following calls to "now" will just use the new (mocked) Date.now + // method directly. + installMockDate(); + now = Date.now; + return Date.now(); + }; + var originalDate = Date; + var originalNow = originalDate.now; + function advanceTimeTo(time) { + timeOffset = time - originalNow(); + } + function installMockDate() { + var NewDate = function Date(Y, M, D, h, m, s, ms) { + if (this instanceof Date) { + // Assume a construct call. + switch (arguments.length) { + case 0: return new originalDate(originalNow() + timeOffset); + case 1: return new originalDate(Y); + case 2: return new originalDate(Y, M); + case 3: return new originalDate(Y, M, D); + case 4: return new originalDate(Y, M, D, h); + case 5: return new originalDate(Y, M, D, h, m); + case 6: return new originalDate(Y, M, D, h, m, s); + default: return new originalDate(Y, M, D, h, m, s, ms); + } + } + return new originalDate(originalNow() + timeOffset).toString(); + }; + NewDate.UTC = originalDate.UTC; + NewDate.parse = originalDate.parse; + NewDate.now = function now() { return originalNow() + timeOffset; }; + NewDate.prototype = originalDate.prototype; + originalDate.prototype.constructor = NewDate; + Date = NewDate; + } + + // Heap priority queue with key index. + // Each entry is list of [timeout, callback1 ... callbackn]. + var timerHeap = []; + var timerIndex = {}; + function addDelayedTimer(f, ms) { + var timeout = now() + ms; + var timerList = timerIndex[timeout]; + if (timerList == null) { + timerList = [timeout, f]; + timerIndex[timeout] = timerList; + var index = timerHeap.length; + timerHeap.length += 1; + bubbleUp(index, timeout, timerList); + } else { + timerList.push(f); + } + } + + function nextDelayedTimerQueue() { + if (timerHeap.length == 0) return null; + var result = timerHeap[0]; + var last = timerHeap.pop(); + if (timerHeap.length > 0) { + bubbleDown(0, last[0], last); + } + return result; + } + + function bubbleUp(index, key, value) { + while (index != 0) { + var parentIndex = (index - 1) >> 1; + var parent = timerHeap[parentIndex]; + var parentKey = parent[0]; + if (key > parentKey) break; + timerHeap[index] = parent; + index = parentIndex; + } + timerHeap[index] = value; + } + + function bubbleDown(index, key, value) { + while (true) { + var leftChildIndex = index * 2 + 1; + if (leftChildIndex >= timerHeap.length) break; + var minChildIndex = leftChildIndex; + var minChild = timerHeap[leftChildIndex]; + var minChildKey = minChild[0]; + var rightChildIndex = leftChildIndex + 1; + if (rightChildIndex < timerHeap.length) { + var rightChild = timerHeap[rightChildIndex]; + var rightKey = rightChild[0]; + if (rightKey < minChildKey) { + minChildIndex = rightChildIndex; + minChild = rightChild; + minChildKey = rightKey; + } + } + if (minChildKey > key) break; + timerHeap[index] = minChild; + index = minChildIndex; + } + timerHeap[index] = value; + } + + function addInterval(f, ms) { + var id = timerIdCounter++; + function repeat() { + // Reactivate with the same id. + repeat.$timerId = id; + timerIds[id] = repeat; + addDelayedTimer(repeat, ms); + f(); + } + repeat.$timerId = id; + timerIds[id] = repeat; + addDelayedTimer(repeat, ms); + return id; + } + + function cancelTimer(id) { + var f = timerIds[id]; + if (f == null) return; + clearTimerId(f, id); + } + + function clearTimerId(f, id) { + f.$timerId = undefined; + delete timerIds[id]; + } + + function eventLoop(action) { + while (action) { + try { + action(); + } catch (e) { + if (typeof onerror == "function") { + onerror(e, null, -1); + } else { + throw e; + } + } + action = nextEvent(); + } + } + + // Global properties. "self" refers to the global object, so adding a + // property to "self" defines a global variable. + self.dartMainRunner = function(main, args) { + // Initialize. + var action = function() { main(args); } + eventLoop(action); + }; + self.setTimeout = addTimer; + self.clearTimeout = cancelTimer; + self.setInterval = addInterval; + self.clearInterval = cancelTimer; + self.scheduleImmediate = addTask; + self.self = self; +})(self); diff --git a/benchmark/src/benchmark.dart b/benchmark/src/benchmark.dart deleted file mode 100644 index 67a3c3d..0000000 --- a/benchmark/src/benchmark.dart +++ /dev/null @@ -1,55 +0,0 @@ -// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code -// is governed by a BSD-style license that can be found in the LICENSE file. - -// Author: Paul Brauner (polux@google.com) - -part of map_bench; - -class Benchmark { - - int size; - - Benchmark([this.size = 1000]); - - static warmup() { - new Benchmark(1000).bench(); - } - - Map bench() { - var res = {}; - res["Linked List"] = _bench(() => new SimplePersistentMap()); - res["Mutable Map"] = _bench(() => new SimplePersistentMap2()); - res["Hash Trie"] = _bench(() => new PersistentMap()); - return res; - } - - int _bench(PersistentMap empty()) { - final stopwatch = new Stopwatch(); - stopwatch.start(); - - PersistentMap map = empty(); - for (int i = 0; i < size; i++) { - map = map.insert("key$i", "foo", (String x, String y) => x + y); - map = map.insert("key$i", "bar", (String x, String y) => x + y); - } - - for (int i = size * 2; i >= 0; i--) { - map.lookup("key$i"); - } - for (int i = 0; i <= size * 2; i++) { - map.lookup("key$i"); - } - - PersistentMap saved = map; - for (int i = size * 2; i >= 0; i--) { - map = map.delete("key$i"); - } - map = saved; - for (int i = 0; i < size * 2; i++) { - map = map.delete("key$i"); - } - - stopwatch.stop(); - return stopwatch.elapsedMicroseconds; - } -} diff --git a/benchmark/src/simple_map_1.dart b/benchmark/src/simple_map_1.dart deleted file mode 100644 index 0dedcca..0000000 --- a/benchmark/src/simple_map_1.dart +++ /dev/null @@ -1,97 +0,0 @@ -// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code -// is governed by a BSD-style license that can be found in the LICENSE file. - -// Author: Paul Brauner (polux@google.com) - -part of map_bench; - -/** - * Naive implementation of PersistentMap using a [LinkedList] of [Pair]s. - */ -class SimplePersistentMap extends PersistentMapBase { - final LinkedList> _list; - - bool get isEmpty => _list.isNil; - - SimplePersistentMap._internal(this._list); - factory SimplePersistentMap() => - new SimplePersistentMap._internal(new Nil()); - - PersistentMap insert(K key, V value, [V combine(V x, V y)]) { - combine = (combine != null) ? combine : (V x, V y) => y; - LinkedList> newList() { - LinkedListBuilder> builder = - new LinkedListBuilder>(); - LinkedList> it = _list; - while (it.isCons) { - Cons> cons = it.asCons; - Pair elem = cons.elem; - if (elem.fst == key) { - builder.add(new Pair(key, combine(elem.snd, value))); - return builder.build(cons.tail); - } - builder.add(elem); - it = cons.tail; - } - builder.add(new Pair(key, value)); - return builder.build(); - } - return new SimplePersistentMap._internal(newList()); - } - - PersistentMap delete(K key) => - new SimplePersistentMap._internal(_list.strictWhere((p) => p.fst != key)); - - Option lookup(K key) { - LinkedList> it = _list; - while (it.isCons) { - Cons> cons = it.asCons; - Pair elem = cons.elem; - if (elem.fst == key) return new Option.some(elem.snd); - it = cons.tail; - } - return new Option.none(); - } - - PersistentMap mapValues(f(V)) => - new SimplePersistentMap._internal( - _list.strictMap((p) => new Pair(p.fst, f(p.snd)))); - - forEachKeyValue(f(K, V)) { - Map tmp = new Map(); - _list.foreach((pair) { - if (!tmp.containsKey(pair.fst)) - tmp[pair.fst] = pair.snd; - }); - tmp.forEach(f); - } - - String toString() => _list.toString(); - - int get length => toMap().length; - - PersistentMap union(PersistentMap other, [V combine(V x, V y)]) { - throw new UnsupportedError("union is not supported"); - } - - PersistentMap intersection(PersistentMap other, - [V combine(V x, V y)]) { - throw new UnsupportedError("intersection is not supported"); - } - - PersistentMap adjust(K key, V update(V)) { - throw new UnsupportedError("adjust is not supported"); - } - - Iterator> get iterator { - final res = >[]; - this.forEachKeyValue((k, v) { - res.add(new Pair(k, v)); - }); - return res.iterator; - } - - Pair pickRandomEntry([Random random]) { - throw new UnsupportedError(""); - } -} diff --git a/benchmark/src/simple_map_2.dart b/benchmark/src/simple_map_2.dart deleted file mode 100644 index 9ef1b46..0000000 --- a/benchmark/src/simple_map_2.dart +++ /dev/null @@ -1,86 +0,0 @@ -// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code -// is governed by a BSD-style license that can be found in the LICENSE file. - -// Author: Paul Brauner (polux@google.com) - -part of map_bench; - -/** - * Naive implementation of PersistentMap dart:core [Map]s. - */ -class SimplePersistentMap2 extends PersistentMapBase { - final Map _map; - - bool get isEmpty => _map.isEmpty; - - SimplePersistentMap2._internal(this._map); - factory SimplePersistentMap2() => - new SimplePersistentMap2._internal(new Map()); - - PersistentMap insert(K key, V value, [V combine(V x, V y)]) { - combine = (combine != null) ? combine : (V x, V y) => y; - Map newmap = new Map.from(_map); - newmap[key] = _map.containsKey(key) ? combine(_map[key], value) : value; - return new SimplePersistentMap2._internal(newmap); - } - - PersistentMap delete(K key) { - Map newmap = new Map.from(_map); - newmap.remove(key); - return new SimplePersistentMap2._internal(newmap); - } - - Option lookup(K key) { - if (_map.containsKey(key)) { - return new Option.some(_map[key]); - } else { - return new Option.none(); - } - } - - PersistentMap mapValues(f(V)) { - Map newmap = new Map.from(_map); - _map.forEach((K key, V value) { - newmap[key] = f(value); - }); - return new SimplePersistentMap2._internal(newmap); - } - - PersistentMap adjust(K key, V update(V)) { - if (_map.containsKey(key)) { - Map newmap = new Map.from(_map); - newmap[key] = update(_map[key]); - return new SimplePersistentMap2._internal(newmap); - } - return this; - } - - void forEachKeyValue(f(K, V)) { - _map.forEach(f); - } - - String toString() => _map.toString(); - - int get length => _map.length; - - PersistentMap union(PersistentMap other, [V combine(V x, V y)]) { - throw new UnsupportedError("union is not supported"); - } - - PersistentMap intersection(PersistentMap other, - [V combine(V x, V y)]) { - throw new UnsupportedError("intersection is not supported"); - } - - Iterator> get iterator { - final res = >[]; - this.forEachKeyValue((k, v) { - res.add(new Pair(k, v)); - }); - return res.iterator; - } - - Pair pickRandomEntry([Random random]) { - throw new UnsupportedError(""); - } -} diff --git a/benchmark/vector_memory/makefile b/benchmark/vector_memory/makefile new file mode 100644 index 0000000..fb0f407 --- /dev/null +++ b/benchmark/vector_memory/makefile @@ -0,0 +1,56 @@ +template_size = 16 + +all: clean run + +define RUNNER +function dartMainRunner(main, args){ + main(process.argv.slice(2)); +} +endef + +export RUNNER + +define INTRO +- - - - - - - - +Using (flat) list of size ${template_size}. Benchmark internally counts number of objects that fit +into 1GB heap space. Output is in bytes per one storage place. Instances of values are shared +between multiple copies of the structure we create, therefore their size do not affect the result +(much). +- - - - - - - - +endef + +export INTRO + +build/vector_memory.js: vector_memory.dart + mkdir -p build + dart2js -o build/tmp.js vector_memory.dart + cat ../preamble/d8.js >> build/vector_memory.js + cat build/tmp.js >> build/vector_memory.js + echo "$$RUNNER" >> build/vector_memory.js + + +clean: + rm -rf build + + +run: build/vector_memory.js + + @echo "$$INTRO" + + @echo -n "NodeJS list: " + @ (! node --max-old-space-size=1024 build/vector_memory.js ${template_size} list > build/temp.out) 2>/dev/null + @tail -1 build/temp.out + + @echo -n "DartVM persistent: " + @dart --old_gen_heap_size=1024 vector_memory.dart ${template_size} persistent > build/temp.out 2>/dev/null + @tail -1 build/temp.out + + @echo -n "DartVM list: " + @dart --old_gen_heap_size=1024 vector_memory.dart ${template_size} list > build/temp.out 2>/dev/null + @tail -1 build/temp.out + + @echo -n "NodeJS persistent: " + @ (! node --max-old-space-size=1024 build/vector_memory.js ${template_size} persistent > build/temp.out) 2>/dev/null + @tail -1 build/temp.out + + diff --git a/benchmark/vector_memory/vector_memory.dart b/benchmark/vector_memory/vector_memory.dart new file mode 100644 index 0000000..196644b --- /dev/null +++ b/benchmark/vector_memory/vector_memory.dart @@ -0,0 +1,44 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +library vector_memory; + +import 'package:vacuum_persistent/persistent.dart'; + +List template = []; + +// here we will store the data to prevent it from garbage collection +List data = []; + +var creators = { + + "persistent": () => new PVec.from(template), + + "list": () => new List.from(template), +}; + +void run(int template_size, String mode) { + + template.addAll(new List.generate(template_size, (i)=>"$i".padLeft(8))); + + int allocated = 0; + for(bool go = true; go; allocated++){ + try{ + go = false; + var a = creators[mode](); + data.add(a); + go = true; + print(1073741824.0 / allocated / template_size); + } catch(e) { + data = null; + } + } +} + +main(List args){ + + run(int.parse(args[0]), args[1]); +} diff --git a/benchmark/vector_speed/benchmarks.dart b/benchmark/vector_speed/benchmarks.dart new file mode 100644 index 0000000..2f0080f --- /dev/null +++ b/benchmark/vector_speed/benchmarks.dart @@ -0,0 +1,79 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of vector_speed; + +class WriteBenchmark extends BenchmarkBase{ + + final int size; + final BenchmarkInterface object; + + + WriteBenchmark(size, object, name): + size = size, + object = object, + super("Writing $name($size)"); + + + void run(){ + + object.create(); + + for (int i = 0; i < size; i++) { + object.push("${i}".padLeft(8,"0")); + } + + for (int i = size-1; i >= 0; i--) { + object.set(i, "${i}".padLeft(8,"_")); + } + + for (int i = 0; i < size; i++) { + object.set(i, "${i}".padLeft(8,"x")); + } + + object.save(); + for (int i = size-1 ; i >= 0; i--) { + object.pop(); + } + + object.restore(); + for (int i = 0; i < size; i++) { + object.pop(); + } + } +} + +class ReadBenchmark extends BenchmarkBase{ + + final int size; + final BenchmarkInterface object; + + + ReadBenchmark(size, object, name): + size = size, + object = object, + super("Reading $name($size)"); + + void setup(){ + + object.create(); + + for (int i = 0; i < size; i++) { + object.push("${i}".padLeft(8,"0")); + } + } + + void run(){ + + for (int i = size-1; i >= 0; i--) { + object.get(i); + } + + for (int i = 0; i < size; i++) { + object.get(i); + } + } +} diff --git a/benchmark/vector_speed/interface.dart b/benchmark/vector_speed/interface.dart new file mode 100644 index 0000000..2dc4d81 --- /dev/null +++ b/benchmark/vector_speed/interface.dart @@ -0,0 +1,37 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of vector_speed; + +abstract class BenchmarkInterface{ + + void create(); + void push(E value); + void set(int index, E value); + void get(int index); + void pop(); + + void save(); + void restore(); +} + + +abstract class EncapsulatingInterface + extends BenchmarkInterface{ + + T object = null; + T object_copy = null; + + T _copy(); + + save(){ + object_copy = _copy(); + } + + restore(){ + object = object_copy; + } +} diff --git a/benchmark/vector_speed/interface_impl.dart b/benchmark/vector_speed/interface_impl.dart new file mode 100644 index 0000000..618015b --- /dev/null +++ b/benchmark/vector_speed/interface_impl.dart @@ -0,0 +1,71 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of vector_speed; + +class PersistentVectorInterface + extends EncapsulatingInterface>{ + + create() => object = new PVec(); + + push(E value) => + object = object.push(value); + + get(int index) => object.get(index); + + set(int index, E value) => object = object.set(index, value); + + pop() => object = object.pop(); + + _copy() => object; +} + + +class TransientVectorInterface + extends EncapsulatingInterface>{ + + create() => object = new PVec().asTransient(); + + push(E value) => + object.doPush(value); + + get(int index) => object.get(index); + + set(int index, E value) => object.doSet(index, value); + + pop() => object.doPop(); + + _copy(){ + return new PVec.from(object).asTransient(); + } +} + + +class ListInterface + extends EncapsulatingInterface>{ + + create() => object = []; + + push(E value) => + object.add(value); + + get(int index) => object[index]; + + set(int index, E value) => object[index] = value; + + pop() => object.removeLast(); + + _copy(){ + return new List.from(object); + } +} + + + + + + + diff --git a/benchmark/vector_speed/makefile b/benchmark/vector_speed/makefile new file mode 100644 index 0000000..446c05b --- /dev/null +++ b/benchmark/vector_speed/makefile @@ -0,0 +1,29 @@ +all: clean run + + +build/vector_speed.js: vector_speed.dart + mkdir -p build + dart2js -o build/tmp.js vector_speed.dart + cat ../preamble/d8.js >> build/vector_speed.js + cat build/tmp.js >> build/vector_speed.js + + dart2js -o build/vector_speed.js vector_speed.dart + + +clean: + rm -rf build + + +run: build/vector_speed.js + + @echo "Performing several get, set, push and pop operations on flat vectors." + @echo "The number of operations depends on the map size." + + @echo "--------" + @echo "DartVM:" + @dart vector_speed.dart + + @echo "--------" + @echo "NodeJS:" + @node build/vector_speed.js + diff --git a/benchmark/vector_speed/vector_speed.dart b/benchmark/vector_speed/vector_speed.dart new file mode 100644 index 0000000..56634a9 --- /dev/null +++ b/benchmark/vector_speed/vector_speed.dart @@ -0,0 +1,36 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +library vector_speed; + +import 'package:vacuum_persistent/persistent.dart'; +import 'package:benchmark_harness/benchmark_harness.dart'; + +part 'benchmarks.dart'; +part 'interface.dart'; +part 'interface_impl.dart'; + +var interfaces = { + "PersistentVector": () => new PersistentVectorInterface(), + "TransientVector": () => new TransientVectorInterface(), + "List": () => new ListInterface(), +}; + +void main() { + + for (int n in [1,10,100,1000,10000]) { + for (String name in interfaces.keys){ + new ReadBenchmark(n, interfaces[name](), name).report(); + } + } + + for (int n in [1,10,100,1000,10000]) { + for (String name in interfaces.keys){ + new WriteBenchmark(n, interfaces[name](), name).report(); + } + } + +} diff --git a/changes_2_0.md b/changes_2_0.md new file mode 100644 index 0000000..f741310 --- /dev/null +++ b/changes_2_0.md @@ -0,0 +1,9 @@ +- memory footprint reduced with a factor of 15 (wait what? Was the old implementation so + ineffective? Or the new one is so cool? The truth is: both. Check out benchmarks) + +- changes in API, most notably PersistentMap -> PMap, PersistentVector -> PVec + +- more effective == and != on PMap + +- deleted several classes, the whole class/interface hierarchy becomes much simpler (although little bit dirtier; some performance-motivated compromises were introduced) + diff --git a/example/example.dart b/example/example.dart new file mode 100644 index 0000000..48cb0cf --- /dev/null +++ b/example/example.dart @@ -0,0 +1,41 @@ +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Author: Paul Brauner (polux@google.com) + +library map_example; + +import 'package:vacuum_persistent/persistent.dart'; + +main() { + + PMap map = new PMap.from({"a":1, "b":2}); + + print(map["a"]); // 1 + print(map.get("b")); // 2 + print(map.get("c", ":(")); // :( + + print(map.assoc("c", 3)); // {a: 1, b: 2, c: 3} + print(map.assoc("d", 4)); // {a: 1, b: 2, d: 4} + + print(map.update("a", (x) => x+1)); // {a: 2, b: 2} + print(map.delete("b")); // {a: 1} + print(map.delete("not-a-key", missingOk: true)); // does not throw + + print(map); // {a: 1, b: 2} (i.e. the map stays unchanged) + + // Transiency: + + final vec = new PVec.from(["a", "b"]); + + print(vec.push("c")); // (a, b, c) + print(vec.push("d")); // (a, b, d) + + var temp = vec.asTransient(); + temp.doPush("c"); + temp.doPush("d"); + temp[0] = "A"; + final vec2 = temp.asPersistent(); + print(vec2); // (A, b, c, d) + +} diff --git a/example/map_example.dart b/example/map_example.dart deleted file mode 100644 index 944c667..0000000 --- a/example/map_example.dart +++ /dev/null @@ -1,39 +0,0 @@ -// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code -// is governed by a BSD-style license that can be found in the LICENSE file. - -// Author: Paul Brauner (polux@google.com) - -library map_example; - -import 'package:persistent/persistent.dart'; - -main() { - final emptyMap = new PersistentMap(); - final m1 = emptyMap.insert('a', 1).insert('b', 2); - final m2 = new PersistentMap.fromMap({'a': 3, 'c': 4}); - - print(m1); // {a: 1, b: 2} - print(m2); // {c: 4, a: 3} - print(m1.lookup('a')); // Option.some(1) - print(m1.lookup('c')); // Option.none() - - final m3 = m1.delete('a'); - print(m1); // {a: 1, b: 2} - print(m3); // {b: 2} - - final m4 = m1.union(m2, (n,m) => n + m); - print(m4); // {c: 4, a: 4, b: 2} - - final m5 = m1.mapValues((n) => n + 1); - print(m5); // {a: 2, b: 3} - - final m6 = m1.adjust('a', (n) => n + 1); - print(m6); // {a: 2, b: 2} - - for (final pair in m4) { - print(pair); // Pair(a, 4), Pair(c, 4), Pair(b, 2) - } - - print(m4.keys.toList()); // [a, c, b] - print(m4.values.toList()); // [4, 4, 2] -} diff --git a/example/overall_example.dart b/example/overall_example.dart new file mode 100644 index 0000000..d4a2322 --- /dev/null +++ b/example/overall_example.dart @@ -0,0 +1,81 @@ +// Copyright (c) 2014, VaccumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +library overall_example; + +import 'package:vacuum_persistent/persistent.dart'; + +example(){ + var couple = new PMap.fromMap({'father': 'Homer', 'mother': 'Marge'}); + var withChild = couple.assoc('boy', 'Bart'); + print(couple); // {mother: Marge, father: Homer} + print(withChild); // {boy: Bart, mother: Marge, father: Homer} +} + +example2(){ + // deeply persist the structure of Maps and Lists + var a = per({[1,2]: 'tower', [1,3]: 'water'}); + var b = per({[1,2]: 'tower', [1,3]: 'water'}); + assert(a==b); + // kids, don't try this with standard List, it ain't work + print(a[per([1, 2])]); // prints hello +} + +main() { + + // Persistency: + + final map1 = new PMap.fromMap({"a":1, "b":2}); + final map2 = new PMap.fromMap({"b":3, "c":4}); + + print(map1["a"]); // 1 + print(map1.get("b")); // 2 + print(map1.get("c", ":(")); // :( + + print(map1.assoc("c", 3)); // {a: 1, b: 2, c: 3} + print(map1.assoc("d", 4)); // {a: 1, b: 2, d: 4} + + final map3 = map2._update("c", (x) => x+3); + print(map3.delete("b")); // {c: 7} + print(map3.delete("a", missingOk: true)); // {b: 3, c: 7} + + print(map1); // {a: 1, b: 2} + print(map2); // {b: 3, c: 4} + print(map3); // {b: 3, c: 7} + + // Transiency: + + final vector1 = new PVec.from(["x", "y"]); + + print(vector1.push("z")); // (x, y, z) + print(vector1.push("q")); // (x, y, q) + + var temp = vector1.asTransient(); + temp.doPush("z"); + temp.doPush("q"); + temp[1] = "Y"; + final vector2 = temp.asPersistent(); + + final vector3 = vector2.withTransient((TransientVector v){ + v.doSet(2, "Z"); + v.doPop(); + v[0] = "X"; + }); + + print(vector1); // (x, y) + print(vector2); // (x, Y, z, q) + print(vector3); // (X, Y, Z) + + // Features + + print(map1.toList()); // [Pair(a, 1), Pair(b, 2)] + + final set1 = new PersistentSet.from(["a", "b"]); + final set2 = new PersistentSet.from([1, 2, 3]); + print((set1 * set2).toList()); + // [Pair(a, 1), Pair(a, 2), Pair(a, 3), Pair(b, 1), Pair(b, 2), Pair(b, 3)] + +} diff --git a/example/set_example.dart b/example/persistent_set_example.dart similarity index 94% rename from example/set_example.dart rename to example/persistent_set_example.dart index f8bf121..5c6d75a 100644 --- a/example/set_example.dart +++ b/example/persistent_set_example.dart @@ -5,7 +5,7 @@ library map_example; -import 'package:persistent/persistent.dart'; +import 'package:vacuum_persistent/persistent.dart'; main() { final emptySet = new PersistentSet(); diff --git a/haskell-model/ImMap.hs b/haskell-model/ImMap.hs deleted file mode 100644 index 4675d7d..0000000 --- a/haskell-model/ImMap.hs +++ /dev/null @@ -1,151 +0,0 @@ --- Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code --- is governed by a BSD-style license that can be found in the LICENSE file. - --- Author: Paul Brauner (polux@google.com) - -module ImMap ( - Map(), empty, adjust, lookup, insert, delete, unionWith, toList, - fromList, member, size, insertWith -) where - -import qualified SmallMap as IM -import qualified Data.Map as M -- for cheak equality -import Prelude hiding (lookup) -import qualified Prelude as P -import Data.Hashable -import Data.Bits -import Data.Maybe -import Data.List (null, (\\)) -import Control.Arrow((***)) - -data Map k v = EmptyMap - | Leaf Int [(k, v)] - | SubMap (IM.SmallMap (Map k v)) - deriving (Show) - -empty = EmptyMap - --- part n i selects the nth part of i -part n i = (i `shiftR` (n * 5)) .&. 0x1f -hash30 k = (hash k `shiftR` 2) .&. 0x3fffffff - -lookup :: (Eq k, Hashable k) => k -> Map k v -> Maybe v -lookup k m = lookup' 0 m - where - code = hash30 k - lookup' _ EmptyMap = Nothing - lookup' _ (Leaf h assocs) | code == h = P.lookup k assocs - | otherwise = Nothing - lookup' l (SubMap m) = lookup' (l+1) =<< IM.lookup (part l code) m - - -insert :: (Eq k, Hashable k) => k -> v -> Map k v -> Map k v -insert k v m = insertWith const k v m - -insertWith :: (Eq k, Hashable k) => (v -> v -> v) -> k -> v -> Map k v -> Map k v -insertWith f k v m = insertWith' f 0 (hash30 k) [(k,v)] m - -insertWith' :: Eq k - => (v -> v -> v) -- merge function - -> Int -- depth - -> Int -- hashcode - -> [(k, v)] -- assoc list to insert - -> Map k v - -> Map k v -insertWith' f depth code kvs m = insertWith'' depth m - where - insertWith'' _ EmptyMap = Leaf code kvs - insertWith'' 6 (Leaf h assocs) = Leaf code (insertAssocWith f kvs assocs) - insertWith'' l lf@(Leaf code2 assocs) - | code == code2 = Leaf code (insertAssocWith f kvs assocs) - | otherwise = insertWith'' l (SubMap $ IM.singleton (part l code2) lf) - insertWith'' l (SubMap im) = - SubMap $ case IM.lookup i im of - Nothing -> IM.insert i (Leaf code kvs) im - Just m -> IM.insert i (insertWith'' (l+1) m) im - where i = part l code - -insertAssocWith :: Eq k - => (v -> v -> v) -- merge function - -> [(k, v)] -- assoc list to insert - -> [(k, v)] - -> [(k, v)] -insertAssocWith f kvs assocs = foldl iter assocs kvs - where - iter acc (k,v) = insertAssocWith' k v acc - insertAssocWith' k v [] = [(k,v)] - insertAssocWith' k v (p@(k', v') : assocs) - | k == k' = (k, f v v') : assocs - | otherwise = p : insertAssocWith' k v assocs - -delete :: (Eq k, Hashable k) => k -> Map k v -> Map k v -delete k m = delete' 0 m - where - code = hash30 k - delete' _ EmptyMap = EmptyMap - delete' _ lf@(Leaf h assocs) - | h == code = - case filter (\p -> fst p /= k) assocs of - [] -> EmptyMap - assocs' -> (Leaf h assocs') - | otherwise = lf - delete' l m@(SubMap im) = - case IM.lookup i im of - Nothing -> m - Just m' -> build $ case delete' (l+1) m' of - EmptyMap -> IM.delete i im - m'' -> IM.insert i m'' im - where i = part l code - build im = case IM.elems im of - [] -> error "should never happen" - [l@(Leaf _ _)] -> l - _ -> SubMap im - - -unionWith :: Eq k => (v -> v -> v) -> Map k v -> Map k v -> Map k v -unionWith f m1 m2 = unionWith' 0 m1 m2 - where unionWith' _ EmptyMap m = m - unionWith' _ m EmptyMap = m - unionWith' d (SubMap sm1) (SubMap sm2) = SubMap sm - where sm = IM.unionWith (unionWith' (d+1)) sm1 sm2 - unionWith' d m@(SubMap sm) (Leaf h kvs) = insertWith' (flip f) d h kvs m - unionWith' d (Leaf h kvs) m@(SubMap sm) = insertWith' f d h kvs m - unionWith' d (Leaf h1 kvs1) m@(Leaf h2 kvs2) = insertWith' f d h1 kvs1 m - -adjust :: (Eq k, Hashable k) => k -> (v -> v) -> Map k v -> Map k v -adjust k f m = adjust' 0 m - where code = hash30 k - adjust' _ e@EmptyMap = e - adjust' _ l@(Leaf h kvs) | h == code = Leaf h (adjust'' kvs) - | otherwise = l - adjust' d (SubMap sm) = SubMap (IM.adjust (adjust' (d+1)) (part d code) sm) - - adjust'' [] = [] - adjust'' (kv@(k',v):kvs) | k' == k = (k', f v):kvs - | otherwise = kv : adjust'' kvs - -size :: Map k v -> Int -size EmptyMap = 0 -size (Leaf _ kvs) = length kvs -size (SubMap sm) = IM.fold (\m a -> size m + a) 0 sm - -toList EmptyMap = [] -toList (Leaf _ xs) = xs -toList (SubMap sm) = concatMap toList (IM.elems sm) - -member k = isJust . lookup k - -fromList l = foldl (\m (k,v) -> insert k v m) EmptyMap l - -instance Functor (Map k) where - fmap f m = fmap' m - where fmap' EmptyMap = EmptyMap - fmap' (Leaf h kvs) = Leaf h (map (id *** f) kvs) - fmap' (SubMap sm) = SubMap (IM.map fmap' sm) - -instance (Eq k, Eq v) => Eq (Map k v) where - EmptyMap == EmptyMap = True - Leaf h1 kvs1 == Leaf h2 kvs2 = h1 == h2 && eqPermut kvs1 kvs2 - where eqPermut xs ys = null (xs \\ ys) - SubMap sm1 == SubMap sm2 = sm1 == sm2 - _ == _ = False diff --git a/haskell-model/ImTest.hs b/haskell-model/ImTest.hs deleted file mode 100644 index 7107c39..0000000 --- a/haskell-model/ImTest.hs +++ /dev/null @@ -1,72 +0,0 @@ --- Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code --- is governed by a BSD-style license that can be found in the LICENSE file. - --- Author: Paul Brauner (polux@google.com) - -import qualified Data.Map as M -import qualified ImMap as IM - -import Test.QuickCheck -import Control.Applicative -import Data.Hashable -import Control.Arrow((***)) -import Text.Show.Functions - --- a datatype with an imperfect hash function -data Key = Key String Bool - deriving (Show, Eq, Ord) - -instance Hashable Key where - hash (Key s _) = hash s - -instance Arbitrary Key where - arbitrary = Key <$> arbitrary <*> arbitrary - shrink (Key s b) = Key <$> shrink s <*> shrink b - -instance (Ord k, Arbitrary k, Arbitrary v) => Arbitrary (M.Map k v) where - arbitrary = M.fromList <$> arbitrary - shrink = map M.fromList . shrink . M.toList - -from = IM.fromList . M.toList -same im m = M.fromList (IM.toList im) == m - -type Val = Int -type M = M.Map Key Val - -equalsProp :: M -> M -> Bool -equalsProp m1 m2 = (from m1 == from m2) == (m1 == m2) - -insertProp :: M -> Key -> Val -> (Val -> Val -> Val) -> Bool -insertProp m k v f = IM.insertWith f k v (from m) `same` M.insertWith f k v m - -deleteProp :: M -> Key -> Bool -deleteProp m k = IM.delete k (from m) `same` M.delete k m - -lookupProp :: M -> Key -> Bool -lookupProp m k = IM.lookup k (from m) == M.lookup k m - -adjustProp :: M -> Key -> (Val -> Val) -> Bool -adjustProp m k f = IM.adjust k f (from m) `same` M.adjust f k m - -fmapProp :: M -> (Val -> Val) -> Bool -fmapProp m f = fmap f (from m) `same` fmap f m - -sizeProp :: M -> Bool -sizeProp m = IM.size (from m) == M.size m - -unionProp :: M -> M -> (Val -> Val -> Val) -> Bool -unionProp m1 m2 f = - IM.unionWith f (from m1) (from m2) `same` M.unionWith f m1 m2 - -check p = quickCheckWith args p - where args = stdArgs { maxSuccess = 1000 } - -main = do - check equalsProp - check insertProp - check deleteProp - check lookupProp - check adjustProp - check fmapProp - check sizeProp - check unionProp diff --git a/haskell-model/SmallMap.hs b/haskell-model/SmallMap.hs deleted file mode 100644 index 31fc9a6..0000000 --- a/haskell-model/SmallMap.hs +++ /dev/null @@ -1,120 +0,0 @@ --- Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code --- is governed by a BSD-style license that can be found in the LICENSE file. - --- Author: Paul Brauner (polux@google.com) - -module SmallMap ( - SmallMap(), - empty, - singleton, - lookup, - elems, - insert, - delete, - adjust, - map, - unionWith, - fold -) where - -import Prelude hiding (lookup, map) -import qualified Data.Array.IArray as A -import Data.Maybe(catMaybes) -import Data.Bits -import GHC.Int - -import Debug.Trace - -type BitMap = Int -type Mask = Int - -data SmallMap a = SmallMap - { bitmap :: BitMap - , array :: A.Array Int a - } deriving (Eq, Show) - - -popcount x = - let x1 = x - ((x `shiftR` 1) .&. 0x55555555); - x2 = (x1 .&. 0x33333333) + ((x1 `shiftR` 2) .&. 0x33333333); - x3 = (x2 + (x2 `shiftR` 4)) .&. 0x0F0F0F0F; - x4 = x3 + (x3 `shiftR` 8); - x5 = x4 + (x4 `shiftR` 16); - in x5 .&. 0x0000003F; - -mask :: Int -> Mask -mask i = 1 `shiftL` i - -index :: BitMap -> Mask -> Int -index b m = popcount (b .&. (m - 1)) - -set i x a = a A.// [(i, x)] - -insertAt i x a = A.listArray (0, u+1) na - where (_, u) = A.bounds a - na = take i ea ++ [x] ++ drop i ea - ea = A.elems a - -deleteAt i a = A.listArray (0, u-1) na - where (_, u) = A.bounds a - na = take i ea ++ drop (i+1) ea - ea = A.elems a - -empty :: SmallMap a -empty = SmallMap 0 (A.listArray (0, -1) []) - -insert :: Int -> a -> SmallMap a -> SmallMap a -insert i x (SmallMap b a) - | (b .&. msk) == 0 = SmallMap (b .|. msk) (insertAt idx x a) - | otherwise = SmallMap b (set idx x a) - where msk = mask i - idx = index b msk - -singleton :: Int -> a -> SmallMap a -singleton i x = insert i x empty - -lookup :: Int -> SmallMap a -> Maybe a -lookup i (SmallMap b a) - | (b .&. msk) == 0 = Nothing - | otherwise = Just (a A.! idx) - where msk = mask i - idx = index b msk - -delete :: Int -> SmallMap a -> SmallMap a -delete i sm@(SmallMap b a) - | (b .&. msk) == 0 = error "no value at i in delete" - | otherwise = SmallMap (b .&. complement msk) (deleteAt idx a) - where msk = mask i - idx = index b msk - -unionWith :: (a -> a -> a) -> SmallMap a -> SmallMap a -> SmallMap a -unionWith f (SmallMap b1 a1) (SmallMap b2 a2) = SmallMap b a - where b = b1 .|. b2 - b' = b1 .&. b2 - a = go (A.array (0, popcount b - 1) []) 1 0 0 0 - go a m i j c | m > b = a - | b' .&. m /= 0 = go (set c (f (a1 A.! i) (a2 A.! j)) a) - (m `shiftL` 1) (i + 1) (j + 1) (c + 1) - | b1 .&. m /= 0 = go (set c (a1 A.! i) a) - (m `shiftL` 1) (i + 1) j (c + 1) - | b2 .&. m /= 0 = go (set c (a2 A.! j) a) - (m `shiftL` 1) i (j + 1) (c + 1) - | otherwise = go a (m `shiftL` 1) i j c - - -adjust :: (a -> a) -> Int -> SmallMap a -> SmallMap a -adjust f i sm@(SmallMap b a) - | (b .&. msk) == 0 = sm - | otherwise = SmallMap b (set idx (f oldval) a) - where msk = mask i - idx = index b msk - oldval = a A.! idx - -fold :: (a -> b -> b) -> b -> SmallMap a -> b -fold f z = foldr f z . elems - -elems :: SmallMap a -> [a] -elems = A.elems . array - -map :: (a -> b) -> SmallMap a -> SmallMap b -map f (SmallMap b a) = SmallMap b (A.amap f a) diff --git a/lib/persistent.dart b/lib/persistent.dart index 58f0977..1779637 100644 --- a/lib/persistent.dart +++ b/lib/persistent.dart @@ -1,17 +1,26 @@ +// Copyright (c) 2014, VacuumLabs. // Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code // is governed by a BSD-style license that can be found in the LICENSE file. -// Author: Paul Brauner (polux@google.com) +// Authors are listed in the AUTHORS file library persistent; import 'dart:collection'; import 'dart:math'; +import 'package:quiver/core.dart'; +import 'package:quiver/iterables.dart' as quiver; +import 'dart:async'; +part 'src/persistent.dart'; part 'src/map.dart'; part 'src/map_impl.dart'; part 'src/set.dart'; part 'src/set_impl.dart'; part 'src/linked_list.dart'; -part 'src/option.dart'; part 'src/pair.dart'; +part 'src/vector.dart'; +part 'src/vector_impl.dart'; +part 'src/functions.dart'; +part 'src/cursor.dart'; +part 'src/reference.dart'; diff --git a/lib/src/cursor.dart b/lib/src/cursor.dart new file mode 100644 index 0000000..51b6a94 --- /dev/null +++ b/lib/src/cursor.dart @@ -0,0 +1,21 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of persistent; + +class Cursor { + final Reference _ref; + final PVec _path; + + Cursor(Reference this._ref, Iterable path) : + _path = (path is PVec)? path: new PVec.from(path); + + Cursor operator[](val) => new Cursor(_ref, conj(_path, val) as PVec); + + deref([notFound = _none]) => getIn(_ref.deref(), _path, notFound); + + update(f) => _ref.update((_) => updateIn(_ref.deref(), _path, f)); +} diff --git a/lib/src/functions.dart b/lib/src/functions.dart new file mode 100644 index 0000000..135c28a --- /dev/null +++ b/lib/src/functions.dart @@ -0,0 +1,634 @@ +// Copyright (c) 2014, VacuumLabs. +// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code +// is governed by a BSD-style license that can be found in the LICENSE file. + +// Authors are listed in the AUTHORS file + +part of persistent; + +_dispatch(x, {op:"operation", map, vec, set}) { + if (x is PMap) { + if (map != null) return map(); + } else if (x is PVec) { + if (vec != null) return vec(); + } else if (x is PSet) { + if (set != null) return set(); + } + throw new Exception("${x.runtimeType} does not support $op operation"); +} + +_firstP(p) => p is Pair? p.fst : p.first; +_secondP(p) => (p is Pair)? p.snd : p.last; + +/** + * Returns a new collection which is the result of inserting elements to persistent + * [coll] ([PMap]/[PSet]/[PVec]). + * Accepts up to 9 positional elements in one call. If you need to insert + * more elements, call [into] with [List] of elements. + * When conjing to a map as element use [List] of length 2 or [Pair]. + * Inserting element to [PSet], which is already presented, does nothing. + * Inserting element to [PMap] with existing key will overwrite that key. + * + * Examples: + * PersistentVector pv = persist([1, 2]) + * conj(pv, 3, 4, 5); // == persist([1, 2, 3, 4, 5]) + * + * PersistentMap pm = new PersistentMap(); + * conj(pm, ['a', 8]); // == persist({'a': 8}); + * conj(pm, new Pair(['a', 6]), ['b', 10]); // == persist({'a': 6, 'b': 10}) + */ +PersistentCollection conj(PersistentCollection coll, arg0, [arg1 = _none, arg2 = _none, arg3 = _none, arg4 = _none, arg5 = _none, arg6 = _none, arg7 = _none, arg8 = _none, arg9 = _none]) { + var varArgs = [arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9].where((x) => x != _none); + return into(coll, varArgs); +} + +/** + * Returns a new collection which is the result of inserting all elements of [iter] + * to persistent [coll] ([PMap]/[PSet]/[PVec]). + * Inserting element to [PSet], which is already presented, does nothing. + * Inserting element to [PMap] with existing key will overwrite that key. + * + * Examples: + * PersistentVector pv = persist([]); + * insert(pv, [0, 2, 4]); // == persist([0, 2, 4]) + * + * PersistentMap pm1 = persist({'a': 10, 'b': 9}); + * PersistentMap pm2 = persist({'b': 15, 'c': 7}); + * insert(pm1, pm2); // persist({'a':10, 'b': 15, 'c': 7}) + * insert(pm1, [['b', 15], new Pair(c, 7)]); // == persist({'a':10, 'b': 15, 'c': 7}); + * + * PSet perSet = new PSet(); + * into(perSet, [1,2,1,3,2]); // == persist(new Set.from([1, 2, 3])) + */ +PersistentCollection into(PersistentCollection coll, Iterable iter) { + return _dispatch(coll, + op: 'into', + map:()=> (coll as PMap).withTransient((TMap t) => iter.forEach((arg) => t.doAssoc(_firstP(arg), _secondP(arg)))), + vec:()=> (coll as PVec).withTransient((t) => iter.forEach((arg) => t.doPush(arg))), + set:()=> (coll as PSet).withTransient((t) => iter.forEach((arg) => t.doInsert(arg))) + ); +} +/** + * Returns a new collection which is the result of inserting new keys and values + * into [PersistentIndexedCollection] [coll] ([PMap]/[PVec]). + * Accepts up to 9 key:value positional arguments. If you need more arguments use [assocI] with [Iterable]. + * + * Example: + * PersistentMap pm = persist({}); + * assoc(pm, 'a', 5, 'b', 6); // == persist({'a': 5, 'b': 6}) + * + * PersistentVector p = persist([1, 2, 3]); + * assoc(pm, 0, 'a', 2, 'b', 0, 'c'); // == persist(['c', 2, 'b']) + */ +PersistentCollection assoc(PersistentIndexedCollection coll, key0, val0, [ + key1 = _none, val1 = _none, + key2 = _none, val2 = _none, + key3 = _none, val3 = _none, + key4 = _none, val4 = _none, + key5 = _none, val5 = _none, + key6 = _none, val6 = _none, + key7 = _none, val7 = _none, + key8 = _none, val8 = _none, + key9 = _none, val9 = _none + ]) { + var argsAll = [[key0,val0], + [key1,val1], + [key2,val2], + [key3,val3], + [key4,val4], + [key5,val5], + [key6,val6], + [key7,val7], + [key8,val8], + [key9,val9]]; + argsAll.forEach((a) => (a[0] != _none && a[1] ==_none)? + throw new ArgumentError("Even number of keys and values is required") : null); + var varArgs = argsAll.where((x) => x[0]!= _none && x[1] != _none); + return assocI(coll, varArgs); +} + +/** + * Returns a new collection which is the result of adding all elements of [iter] + * into [PersistentIndexedCollection] [coll] ([PMap]/[PVec]). + * Elements of [iter] should be [Pair] or [List] with 2 arguments. + * + * Example: + * PersistentMap pm1 = persist({}); + * assoc(pm1, [['a', 5], new Pair('b', 6)]); // == persist({'a': 5, 'b': 6}) + * + * PersistentVector pm2 = persist([1, 2, 3]); + * assoc(pm2, [[0, 'a'], [2, 'b'], [0, 'c']]); // == persist(['c', 2, 'b']) + */ +PersistentCollection assocI(PersistentIndexedCollection coll, Iterable iter) { + return _dispatch(coll, + op: 'assocI', + map:()=> into(coll, iter), + vec:()=> (coll as PVec).withTransient((t) => iter.forEach((arg) => t[_firstP(arg)] = _secondP(arg))) + ); +} + +/** + * Returns a new [PersistentCollection] which is the result of removing keys from + * persistent [coll] ([PMap]/[PSet]/[PVec]). + * Accepts up to 9 key:value positional arguments. + * If you need more arguments use [dissocI] with [Iterable]. + * + * Removing not existing key do nothing. + * + * Example: + * PersistentMap p = persist({'a': 10, 'b':15, 'c': 17}); + * dissoc(p, 'c', 'b'); // == persist({'a': 10}) + * dissoc(p, 'a'); // == persist({'b': 15, 'c': 17}) + * + * PSet p = persist(['a', 'b', 'c']); + * dissoc(p, 'c', 'b'); // == persist(['a']) + * dissoc(p, 'a'); // == persist(['b', 'c']) + */ +PersistentCollection dissoc(PersistentCollection coll, arg0, [arg1 = _none, arg2 = _none, arg3 = _none, arg4 = _none, arg5 = _none, arg6 = _none, arg7 = _none, arg8 = _none, arg9 = _none]) { + var varArgs = [arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9].where((x) => x != _none); + return dissocI(coll, varArgs); +} + +/** + * Returns a new [Persistent] collection which is the result of removing all + * keys in [iter] from persistent [coll] + * ([PMap]/[PSet]/[PVec]). + * + * Example: + * PersistentMap p = persist({'a': 10, 'b':15, 'c': 17}); + * dissocI(p, ['c', 'b']); // == persist({'a': 10}) + * dissocI(p, ['a']); // == persist({'b': 15, 'c': 17}) +* + * PSet p = persist(['a', 'b', 'c', 'd']); + * dissocI(p, ['a', 'b']); // == persist(['c', 'd']) + * + * PersistentVector p = persist(['a', 'b', 'c', 'd']); + * dissocI(p, [0, 2]); // == persist(['b', 'd']) + * + */ +PersistentCollection dissocI(PersistentCollection coll, Iterable iter){ + return _dispatch(coll, + op: 'dissocI', + map:()=> (coll as PMap).withTransient((TMap t) => + iter.forEach((arg) => t.doDelete(arg, missingOk: true))), + vec:()=> _dissocFromVector(coll as PVec, iter), + set:()=> (coll as PSet).withTransient((TSet t) => + iter.forEach((arg) => t.doDelete(arg, missingOk: true))) + ); +} + +//TODO implement doDelete from Vector more effective +PVec _dissocFromVector(PVec pv, Iterable indexes) { + var r = []; + var indexesSet = indexes.toSet(); + for(int i =0;i< pv.length;i++) (!indexesSet.contains(i))? r.add(pv[i]) : null; + return per(r); +} + + +/** + * Returns a new [PVec] which is the result of removing duplicate elements inside [iter]. + * + * Example: + * PersistentVector p = persist([1, 2, 1, 3, 1, 2]); + * distinct(p); // == persist([1, 2, 3]) + */ +PVec distinct(Iterable iter) { + var r = []; + var s = new Set(); + iter.forEach((i) { + if (!s.contains(i)) { + r.add(i); + s.add(i); + } + }); + return persist(r); +} + +/** + * Returns a new empty collection of the same type as [coll] ([PMap]/[PSet]/[PVec]). + * + * Example: + * PersistentVector pv = persist([1, 2, 3]); + * empty(pv); // == persist([]) + * PersistentMap pv = persist({'a': 10, 'b': 11}); + * empty(pv); // == persist({}) + */ +PersistentCollection empty(PersistentCollection coll) { + return _dispatch(coll, + op: 'empty', + map:()=> new PMap(), + vec:()=> new PVec(), + set:()=> new PSet() + ); +} + +/** + * Returns true if persistent [coll] ([PMap]/[PSet]/[PVec]) contains [key]. + * As for [PSet], true is returned if [key] is element of [coll]. + * As for [PVec], true is returned if [key] is correct index in [coll]. + * + * Example: + * PersistantVector pv = persist([1, 2, 5, 7]); + * hasKey(pv, 0); // == true + * hasKey(pv, 3); // == true + * hasKey(pv, 'a'); // == false + * hasKey(pv, -1); // == false + * + * PersistantSet ps = persist(new Set.from(['a', 'b'])); + * hasKey(ps, 'a'); // == true + * hasKey(ps, 0); // == false + * + * PersistantMap pm = persist({'a' : 10, 'b': 18}); + * hasKey(pm, 'a'); // == true + * hasKey(pm, 'c'); // == false + */ +bool hasKey(PersistentCollection coll, key) { + return _dispatch(coll, + op: 'hasKey', + map:()=> (coll as PMap).containsKey(key), + vec:()=> key >= 0 && key < (coll as PVec).length, + set:()=> (coll as PSet).contains(key) + ); +} + +//TODO Sadly, it doesn't throw any specific Exceptions +/** + * Returns an element of persistent [coll] ([PMap]/[PSet]/[PVec]) stored under + * [key]. [PVec] takes [key] as index. [PSet] takes [key] as element of that set + * and returns it. + * Optional argument [notFound] is returned if [coll] doesn't have that key. + * If you don't specify [notFound] and [key] is missing in [coll], then [Exception] is thrown. + * [null] is a valid value for [notFound] and results in same behaviour as dart maps. + * + * Example: + * PersistantMap pm = persist({'a': 10}); + * get(pm, 'a'); // == 10 + * get(pm, 'b'); // throws ... + * get(pm, 'b', null); // null + * + * PersistentVector = persist([11, 22, 33]); + * get(pv, 1); // == 22 + * get(pv, -1); // throw .. + * get(pv, -1, 10); // == 10 + * + * PSet ps = persist(new Set.from(['a', 'b'])); + * get(ps, 'a'); // == 'a' + * get(ps, 'c'); // throw .. + * get(ps, 'c', 17); // 17 + */ +dynamic get(PersistentCollection coll, key, [notFound = _none]) { + return (coll as dynamic).get(key, notFound); +} + +//TODO Sadly, it doesn't throw any specific Exceptions +/** + * Returns an element form recursive persistent [coll] under path of keys. + * [coll] can contain nested [PMap]s, [PVec]s and [PSet]s. + * Optional argument [notFound] is returned if [coll] doesn't have that key path. + * If you don't specify [notFound] and key path is missing in [coll], then [Exception] is thrown. + * + * Example: + * PersistentMap pm = persist({'a': {'b': 10}}); + * getIn(pm, ['a', 'b']); // == 10 + * getIn(pm, ['a', 'b', 'c']); // throw + * getIn(pm, ['a', 'b', 'c'], null); // null + * + * PersistentVector pv = persist([{'a': 0}, 5]); + * getIn(pv, [0, 'a']); // == 0 + * getIn(pv, [0, 'b']); // throws + * getIn(pv, [0, 'b'], 47); // 47 + */ +getIn(PersistentCollection coll, Iterable keys, [notFound = _none]) { + try { + return _getIn(coll, keys, notFound); + } catch (e){ + throw new ArgumentError("Key path $keys doesn't exist in $coll, $e"); + } +} + +_getIn(PersistentCollection coll, Iterable keys, [notFound = _none]) { + if (keys.length == 0) return coll; + if (keys.length == 1) return get(coll, keys.first, notFound); + return getIn(get(coll, keys.first, persist({})), keys.skip(1), notFound); +} + +/** + * Return [Pair] of [key] and result of [get] ([coll], [key], [notFound]). + * + * Example: + * PersistentMap pm = persist({'a' : 10}); + * find(pm, 'a'); // == new Pair('a', 10); + * find(pm, 'b'); // throw + * find(pm, 'b', 15); // == new Pair('b', 15) + */ +Pair find(PersistentCollection coll, key, [notFound = _none]) { + return new Pair(key, get(coll, key, notFound)); +} + +/** + * Returns a new persistent collection which is the result of [assoc] of [val] under [keys] path in [coll]. + * If key path is not present in collection, then [Exception] is thrown + * + * Example: + * PersistentMap pm = persist({'a': [1, 2, 3]}); + * assocIn(pm, ['a', 0], 17); // == persist({'a': [17, 2, 3]); + * + * PersistenMap pm = persist({'a': {'b': 10}}); + * assocIn(pm, ['a', 'c'], 17); // == persist({'a': {'b': 10, 'c': 17}); + * assocIn(pm, ['a', 'c', 'd'], 17); // throws + */ +dynamic assocIn(PersistentIndexedCollection coll, Iterable keys, val) { + try{ + return _assocIn(coll, keys, val); + } catch (e) { + rethrow; + throw new Exception("Key path $keys doesn't exist in coll, $e"); + } +} + +dynamic _assocIn(PersistentIndexedCollection coll, keys, val) { + if (keys.length == 0) return val; + if (keys.length == 1) { + return assoc(coll, keys.first, persist(val)); + } else { + return assoc(coll, keys.first, _assocIn(get(coll, keys.first), keys.skip(1), val)); + } +} + + +/** + * Returns a new persistent collection which is the result of [assoc] of apllied [f] to value under [keys] path in [coll]. + * If only the last key from path is not present, [f] is called without arguments and result is added to return value. + * If other part of key path is not present in collection, then exception is thrown. + * + * Example: + * PersistentMap pm = persist({'a': {'b': 10}}); + * inc(x) => x+1; + * maybeInc([x]) => (x != null)? x+1 : 0; + * updateIn(pm, ['a', 'b'], inc) // == persist({'a': {'b': 11}}) + * updateIn(pm, ['a', 'c'], inc) // throws + * updateIn(pm, ['a', 'c'], maybeInc) // == persist({'a': {'b': 10, 'c': 0}}) + */ +dynamic updateIn(PersistentIndexedCollection coll, Iterable keys, Function f) { + try{ + return _updateIn(coll, keys, f); + } catch (e) { + rethrow; +// throw new Exception("Key path $keys doesn't exist in coll, $e"); + } +} + +dynamic _updateIn(PersistentIndexedCollection coll, Iterable keys, f) { + if (keys.length == 0) return f(coll); + if (keys.length == 1) { + return assoc(coll, keys.first, hasKey(coll, keys.first)? f(get(coll,keys.first)) : f()); + } else { + return assoc(coll, keys.first, _updateIn(get(coll, keys.first), keys.skip(1), f)); + } +} + +/** + * Take iterable [s0] as keys and [s1] as values and return [PMap] constructed from key/value pairs. + * If they have different length, the longer is trimmed to shorter one. + * + * Example: + * zipmap(['a', 'b'], [1, 2]); // == persist({'a': 1, 'b': 2}) + * zipmap(['a', 'b', 'c'], [1, 2, 3, 4, 5]); // == persist({'a': 1, 'b': 2, 'c': 3}) + * zipmap(['a', 'b', 'c', 'd', 'e'], [1, 2, 3]); // == persist({'a': 1, 'b': 2, 'c': 3}) + */ +PMap zipmap(Iterable s0, Iterable s1) { + var m = {}; + while (s0.isNotEmpty && s1.isNotEmpty) { + m[s0.first] = s1.first; + s0 = s0.skip(1); + s1 = s1.skip(1); + } + return persist(m); +} + +/** + * Returns a new [PVec] as subvector from [vector], starting at index [start] inclusive and + * ending at index [end] excusive. If [end] is not set, the length of [vector] is taken as [end] index. + * If they overlap, empty vector is returned. + * + * Example: + * PersistentVector pv = persist([1, 2, 3, 4]); + * subvec(pv, 0); // == persist([1, 2, 3, 4]) + * subvec(pv, 2); // == persist([3, 4]) + * subvec(pv, 1, 3); // == persist([2, 3]) + * subvec(pv, -1, 1); // == persist([1]) + * subvec(pv, 10, -5); // == persist([]) + */ +PVec subvec(PVec vector, start, [end]) { + if (end == null) end = vector.length; + var numberOfElem = end-start; + if (numberOfElem < 0) numberOfElem = 0; + return persist(vector.skip(start).take(numberOfElem)); +} + +/** + * Returns the number of elements in [coll] ([PMap]/[PSet]/[PVec]). + * + * Example: + * PersistentVector pv = persist([1, 2, 3]); + * count(pv); // == 3 + * + * PersistentMap pm = persist({'a': 10, 'b': 17}); + * count(pm); // == 2 + * + * PSet ps = persist(new Set.from('a')); + * count(ps); // == 1 + */ +num count(PersistentCollection coll) => (coll as Iterable).length; + +/** + * Returns whether [coll] ([PMap]/[PSet]/[PVec]) is empty. + * + * Example: + * PersistentVector pv = persist([1, 2, 3]); + * isEmpty(pv); // == false + * + * PersistentMap pm = persist({}); + * isEmpty(pm); // == true + * + * PSet ps = persist(new Set.from('a')); + * isEmpty(ps); // == false + */ +bool isEmpty(PersistentCollection coll) => (coll as Iterable).isEmpty; + +/** + * Reverse order of iteration on [coll]. + * + * Example: + * PersistentVector pv = persist([1, 2, 3]); + * reverse(pv); // == iterable(1, 2, 3) + */ +Iterable reverse(PersistentCollection coll) => persist((coll as Iterable).toList().reversed); + +/** + * Get iterable from keys of [map] ([PMap]). + * + * Example: + * PersistentMap pm = persist({'a' : 10, 'c': 11}); + * keys(pm); // == iterable('a', 'b') + */ +Iterable keys(PMap map) => map.keys; + +/** + * Get iterable from values of [map] ([PMap]). + * + * Example: + * PersistentMap pm = persist({'a' : 10, 'c': 11}); + * values(pm); // == iterable(10, 11) + */ +Iterable values(PMap map) => map.values; + +/** + * Returns a new [PSet] as result of removing [elem] from [coll]. + * It is an inverted operation to [conj]. + * If element is not in [coll], original [coll] is returned. + * + * Example: + * PSet ps = persist(new Set.from(['a', 'b'])); + * disj(ps, 'a'); // == persist(new Set.from(['b'])); + */ +PSet disj(PSet coll, elem) => coll.delete(elem, missingOk: true); + +/** + * Returns a new [PSet] as union of [s1] and [s2]. + * + * Example: + * PSet ps1 = persist(new Set.from(['a', 'b'])); + * PSet ps1 = persist(new Set.from(['b', 'c'])); + * union(ps1, ps2); // == persist(new Set.from(['a', 'b', 'c'])); + */ +PSet union(PSet s1, PSet s2) => s1.union(s2); + +/** + * Returns a new [PSet] as intersection of [s1] and [s2]. + * + * Example: + * PSet ps1 = persist(new Set.from(['a', 'b'])); + * PSet ps1 = persist(new Set.from(['b', 'c'])); + * intersection(ps1, ps2); // == persist(new Set.from(['b'])); + */ +PSet intersection(PSet s1, PSet s2) => s1.intersection(s2); + +/** + * Returns a new [PSet] as difference of [s1] and [s2]. + * + * Example: + * PSet ps1 = persist(new Set.from(['a', 'b'])); + * PSet ps1 = persist(new Set.from(['b', 'c'])); + * difference(ps1, ps2); // == persist(new Set.from(['a'])); + */ +PSet difference(PSet s1, PSet s2) => s1.difference(s2); + +/** + * Returns true if [PSet] [s1] is a subset of [s2]. + * Inverse operation is [isSuperset]. + * + * Example: + * PSet ps1 = persist(new Set.from(['a', 'b', 'c'])); + * PSet ps1 = persist(new Set.from(['b', 'c'])); + * isSubset(ps2, ps1); // == true + * isSubset(ps1, ps2); // == false + */ +bool isSubset(PSet s1, PSet s2) => intersection(s1,s2) == s1; + + +/** + * Returns true if [PSet] [s1] is a superset of [s2]. + * Inverse operation is [isSubset]. + * + * Example: + * PSet ps1 = persist(new Set.from(['a', 'b', 'c'])); + * PSet ps1 = persist(new Set.from(['b', 'c'])); + * isSubset(ps2, ps1); // == false + * isSubset(ps1, ps2); // == true + */ +bool isSuperset(PSet s1, PSet s2) => isSubset(s2, s1); + +// ------------------- SEQUENCES --------------------- + +first(Iterable i) => i.isEmpty ? null: i.first; + +Iterable rest(Iterable i) => i.skip(1); + +Iterable seq(dynamic c) { + if (c is Iterable) { + return c; + } else if (c is String){ + return c.split(''); + } else if (c is Map) { + return persist(c); + } + throw new ArgumentError("Can't convert ${c.runtimeType} to Iterable"); +} + +cons(dynamic val, dynamic coll) => [[val], seq(coll)].expand((x) => x); + +concatI(Iterable a) => a.map((e) => seq(e)).expand((x) => x); + +concat(arg0, arg1, [arg2 = _none, arg3 = _none, arg4 = _none, arg5 = _none, arg6 = _none, arg7 = _none, arg8 = _none, arg9 = _none]) + => concatI([arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9].where((x) => x != _none)); + +//flatten ?? what to do on maps? sets? + +void forEach(Iterable i, f) => i.forEach(f); + +Iterable map(f, Iterable i) => i.map(f); + +Iterable mapcatI(f, Iterable ii) => concatI(ii.map((i) => map(f, i))); + +mapcat(f, arg0, arg1, [arg2 = _none, arg3 = _none, arg4 = _none, arg5 = _none, arg6 = _none, arg7 = _none, arg8 = _none, arg9 = _none]) + => mapcatI(f, [arg0, arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8, arg9].where((x) => x != _none)); + +Iterable filter(pred, Iterable coll) => coll.where(pred); + +Iterable remove(pred, Iterable coll) => filter((x) => !x, coll); + +//TODO maybe descrive better what it does +/** reduce(f, collection) + * reduce(f, intialValue, collection) + * + * Reduce take function [f] of up to two arguments collection and optionaly + * initial value as second argument. Collection is provided as second argument or + * as third argument if initial value is provided + * + * If called on empty collection f() is returned. + * If called on collection with one item this item is returned. + * If called with at least two elements + * - and initial value is provided then collection is foleded with initial value. + * - with no initial value rest of collection is folded with first element and f. + */ +reduce(f, second, [third = _none]) { + Iterable seq; + var initialVal = _none; + if (third == _none) { + seq = second; + } else { + seq = third; + initialVal = second; + } + if (seq.isEmpty) return f(); + if (rest(seq).isEmpty) return first(seq); + return (initialVal == _none)? rest(seq).fold(first(seq), f) : seq.fold(initialVal, f); +} + +//reduce_kv do we want it ??? + +Iterable take(n, Iterable i) => i.take(n); +Iterable takeWhile(pred, Iterable i) => i.takeWhile(pred); +Iterable drop(n, Iterable i) => i.skip(n); +Iterable dropWhile(n, Iterable i) => i.skipWhile(n); +dynamic some(pred, Iterable i) => i.firstWhere(pred); +bool every(pred, Iterable i) => i.every(pred); + +//sort clash with usefull ?? +//sort_by clash with usefull ?? + +Iterable interpose(val, Iterable i) => rest(i.expand((x) => [val, x])); +//interleave(coll1, coll2, ...) +Iterable repeat(Iterable i) => quiver.cycle(i); diff --git a/lib/src/linked_list.dart b/lib/src/linked_list.dart index 0a9888d..3520b04 100644 --- a/lib/src/linked_list.dart +++ b/lib/src/linked_list.dart @@ -5,12 +5,26 @@ part of persistent; +/** + * Immutable list of elements of type E. All its predecessors must + * be accessed before accessing an element. + * + * Can be either [Cons] or [Nil]. + * [Nil] is an empty list, + * while [Cons] contains the first element (`elem`) + * and the rest of the list (`tail`). + */ abstract class LinkedList implements Iterable { bool get isNil; bool get isCons; + + /// Converts this to [Nil] or returns `null` Nil get asNil; + + /// Converts this to [Cons] or returns `null` Cons get asCons; + /// Passes all the elements of this to [f]. void foreach(f(A)); /// A strict (non-lazy) version of [:map:]. @@ -18,37 +32,56 @@ abstract class LinkedList implements Iterable { /// A strict (non-lazy) version of [:where:]. LinkedList strictWhere(bool f(A)); + + /** + * The equality operator. + * + * Two linked lists are equal if and only if they have same lengths, + * and for each possition, the elements at it are equal. + */ + bool operator==(other); + + // Documentation inherited from Object + int get hashCode; } +/** + * [LinkedList] builder. + * + * Elements are added from the first. + */ class LinkedListBuilder { - LinkedList _first = null; - Cons _last = null; + List _data = []; + /// Adds the next element to the list. void add(E x) { - Cons cons = new Cons(x, null); - if (_first == null) { - _first = cons; - } else { - _last.tail = cons; - } - _last = cons; + _data.add(x); + } + + /// Adds all the elements of [iterable] to the list. + void addAll(Iterable iterable) { + _data.addAll(iterable); } + /** + * Creates a new list prepending so far added elements to the + * optional [tail]. + */ LinkedList build([tail = null]) { if (tail == null) tail = new Nil(); - if (_first == null) { - return tail; - } else { - _last.tail = tail; - return _first; + for (E x in _data.reversed){ + tail = new Cons(x, tail); } + return tail; } } abstract class _LinkedListBase extends IterableBase implements LinkedList { + const _LinkedListBase(); + void foreach(f(A)) { LinkedList it = this; while (!it.isNil) { @@ -89,17 +122,26 @@ class _NilIterator implements Iterator { bool moveNext() => false; } +/** + * Empty [LinkedList] + */ class Nil extends _LinkedListBase { bool get isNil => true; bool get isCons => false; Nil get asNil => this; Cons get asCons => null; + const Nil(); + toString() => "nil()"; int get length => 0; Iterator get iterator => const _NilIterator(); + + bool operator==(other) => other is LinkedList ? other.isNil : false; + + int get hashCode => 0; } class _ConsIterator implements Iterator { @@ -123,13 +165,24 @@ class _ConsIterator implements Iterator { } } +/** + * Nonempty [LinkedList] + */ class Cons extends _LinkedListBase { - int _length = null; + final int length; + final int hashCode; + /// The first element of this final E elem; - LinkedList tail; - Cons(this.elem, this.tail); + /// The rest of this - without the first element + final LinkedList tail; + + Cons(elem, tail): + elem = elem, + tail = tail, + length = tail.length + 1, + hashCode = hash2(elem.hashCode, tail.hashCode); bool get isNil => false; bool get isCons => true; @@ -138,12 +191,22 @@ class Cons extends _LinkedListBase { toString() => "cons($elem, $tail)"; - int get length { - if (_length == null) { - _length = tail.length + 1; + Iterator get iterator => new _ConsIterator(this); + + bool operator==(other){ + if (other is! LinkedList) return false; + if ( !other.isCons + || this.hashCode != other.hashCode + || this.length != other.length + ) return false; + var x = this; + var y = other; + while(x.isCons){ + if(x.elem != y.elem) return false; + x = x.tail; + y = y.tail; } - return _length; + return true; } - Iterator get iterator => new _ConsIterator(this); } diff --git a/lib/src/map.dart b/lib/src/map.dart index 7f7e33a..13144bd 100644 --- a/lib/src/map.dart +++ b/lib/src/map.dart @@ -1,110 +1,206 @@ +// Copyright (c) 2014, VacuumLabs. // Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code // is governed by a BSD-style license that can be found in the LICENSE file. -// Authors: -// Paul Brauner (polux@google.com) -// Rafael Brandão (rafa.bra@gmail.com) +// Authors are listed in the AUTHORS file part of persistent; /** - * An immutable map, binding keys of type [K] to values of type [V]. Null values - * are supported but null keys are not. + * The interface is defined mainly for documenting read-functions of PersistentMap + * and TMap; the interface is not supposed to be actually used + * (as you usually know, which Map you are using). + * + * [ReadMap] binds keys of type [K] to values of type [V]. Null + * values are supported but null keys are not. + * + * There is no default implementation of [ReadMap], since it just + * specifies common interface of [PMap] and [TMap]. * * In all the examples below `{k1: v1, k2: v2, ...}` is a shorthand for - * `PersistentMap.fromMap({k1: v1, k2: v2, ...})`. + * `new PersistentMap.fromMap({k1: v1, k2: v2, ...})`. */ -abstract class PersistentMap implements Iterable> { +abstract class ReadMap implements Iterable> { - /** Creates an empty [PersistentMap] using its default implementation. */ - factory PersistentMap() => new _EmptyMap(); + /** + * Returns the value bound to [key]. + * + * If [key] is not bound, [notFound] is returned; if [notFound] is not set, Exception + * is thrown. + */ + V get(K key, [V notFound]); /** - * Creates an immutable copy of [map] using the default implementation of - * [PersistentMap]. - */ - factory PersistentMap.fromMap(Map map) { - PersistentMap result = new _EmptyMap(); - map.forEach((K key, V value) { - result = result.insert(key, value); - }); - return result; - } + * Returns the value bound to [key]. + * + * Throws exception if [key] is not bound. + */ + V operator [](K key); + + /** + * Evaluates `f(key, value)` for each (`key`, `value`) pair in `this`. + */ + void forEachKeyValue(f(K key, V value)); + + /// Returns a mutable copy of `this`. + Map toMap(); + + /// The keys of `this`. + Iterable get keys; + + /// Returns true if contains [key] + bool containsKey(K key); + + /// Returns true if contains [key] + bool hasKey(K key); + + /// The values of `this`. + Iterable get values; + + /// An iterator through the entries of `this`. + Iterator> get iterator; + + /// The number of entries of `this`. + int get length; +} + +/** + * A persistent map, binding keys of type [K] to values of type [V]. Null + * values are supported but null keys are not. + * + * Persistent data structure is an immutable structure, that provides effective + * creation of slightly mutated copies. + * + * In all the examples below `{k1: v1, k2: v2, ...}` is a shorthand for + * `new PersistentMap.fromMap({k1: v1, k2: v2, ...})`. + */ + +abstract class PMap implements ReadMap, PersistentIndexedCollection { + + /** Creates an empty [PMap] using its default implementation. */ + factory PMap() => new _Leaf.empty(null); /** - * Creates a [PersistentMap] from an [Iterable] of [Pair]s using the default - * implementation of [PersistentMap]. + * dispatches to either PMap.fromMap or PMap.fromPairs */ - factory PersistentMap.fromPairs(Iterable> pairs) { - PersistentMap result = new _EmptyMap(); - pairs.forEach((pair) { - result = result.insert(pair.fst, pair.snd); - }); - return result; + factory PMap.from(dynamic source) { + if (source is Map) { + return new PMap.fromMap(source); + } else { + if (source is Iterable) { + if (source.isEmpty) { + return new PMap(); + } else { + if (source.first is Pair) { + return new PMap.fromPairs(source); + } + } + } + } + throw new Exception('cannot construct PMap from ${source.runtimeType} ($source)'); } + /** + * Creates an immutable copy of [map] using the default implementation of + * [PMap]. + */ + factory PMap.fromMap(Map map) => new _Node.fromMap(map); + + /** + * Creates a [PMap] from an [Iterable] of [Pair]s using the default + * implementation of [PMap]. + */ + factory PMap.fromPairs(Iterable> pairs) => new _Node.fromPairs(pairs); + + /** + * The equality operator. + * + * Two persistent maps are equal if and only if their sets of keys are equal, + * and the equal keys are bound to the equal values. + * + * Two sets of keys are equal if and only if for each key exists + * an equal key in the other set. + */ + bool operator== (other); + + /* + * The documentation is inherited from the Object + */ + int get hashCode; + /** * Returns a new map identical to `this` except that it binds [key] to * [value]. * * If [key] was bound to some `oldvalue` in `this`, it is nevertheless bound - * to [value] in the new map. If [key] was bound to some `oldvalue` in `this` - * and if [combine] is provided then [key] it is bound to - * `combine(oldvalue, value)` in the new map. + * to [value] in the new map. * - * {'a': 1}.insert('b', 2) == {'a': 1, 'b', 2} - * {'a': 1, 'b': 2}.insert('b', 3) == {'a': 3, 'b', 3} - * {'a': 1, 'b': 2}.insert('b', 3, (x,y) => x - y) == {'a': 3, 'b', -1} + * {'a': 1}.assoc('b', 2) == {'a': 1, 'b': 2} + * {'a': 1, 'b': 2}.assoc('b', 3) == {'a': 3, 'b': 3} */ - PersistentMap - insert(K key, V value, [V combine(V oldvalue, V newvalue)]); + PMap assoc(K key, V value); /** * Returns a new map identical to `this` except that it doesn't bind [key] * anymore. * + * If [key] is not bound and [missingOk] is not `true`, an Exception is thrown. + * If [key] is not bound and [missingOk] is specified as `true`, + * the same map is returned. [missingOk] defaults to `false`. + * * {'a': 1, 'b': 2}.delete('b') == {'a': 1} - * {'a': 1}.delete('b') == {'a': 1} + * {'a': 1}.delete('b') // throws an Exception */ - PersistentMap delete(K key); + PMap delete(K key, {bool missingOk: false}); /** - * Looks up the value possibly bound to [key] in `this`. Returns - * `Option.some(value)` if it exists, `Option.none()` otherwise. + * Returns a new map identical to `this` except that the value it possibly + * binds to [key] has been adjusted by [f]. * - * {'a': 1}.lookup('b') == Option.none() - * {'a': 1, 'b': 2}.lookup('b') == Option.some(2) - */ - Option lookup(K key); - - /** - * Returns the value for the given [key] or [:null:] if [key] - * is not in the map. + * [f] should have one of the following signatures: V f(V value), V f([V value]) + * + * If [key] is not bound, [f] with no arguments will be called, and the result + * will be associated with [key]. If [key] is not bound and [f] cannot take no arguments, + * an Exception will be thrown. + * + * {'a': 1, 'b': 2}.update('b', (x) => x + 1) == {'a': 1, 'b': 3} + * {'a': 1}.update('b', (x) => x + 1) // throws + * {'a': 2}.update('b', ([x]) => x == null ? 0 : x + 1) == {'a': 2, 'b': 0} */ - V operator [](K key); +// PersistentMap update(K key, dynamic f); /** - * Evaluates `f(key, value)` for each (`key`, `value`) pair in `this`. + * Returns a new map identical to `this` where each value has been updated by + * [f]. + * + * {'a': 1, 'b': 2}.mapValues((x) => x + 1) == {'a': 2, 'b': 3} + * {}.mapValues((x) => x + 1) == {} */ - void forEachKeyValue(f(K key, V value)); +// ReadMap mapValues(f(V value)); /** - * Returns a new map identical to `this` except that the value it possibly - * binds to [key] has been adjusted by [update]. + * Returns a transient copy of `this`. * - * {'a': 1, 'b': 2}.adjust('b', (x) => x + 1) == {'a', 1, 'b', 3} - * {'a': 1}.adjust('b', (x) => x + 1) == {'a', 1} + * This is usually called to make many changes and + * then create a new [PMap]. + * + * var persistent1 = new PersistentMap.from({'a':1}); + * var transient = persistent1.asTransient(); + * transient.doAssoc({'b':2}); + * var persistent2 = new transient.asPersistent(); */ - PersistentMap adjust(K key, V update(V value)); + TMap asTransient(); /** - * Returns a new map identical to `this` where each value has been updated by - * [f]. + * Creates a transient copy of `this`, lets it to be modified by [change] + * and returns a persistent result. * - * {'a': 1, 'b': 2}.mapValues((x) => x + 1) == {'a', 2, 'b', 3} - * {}.mapValues((x) => x + 1) == {} + * var persistent1 = new PersistentMap.from({'a':1}); + * var persistent2 = persistent1.withTransient((m){ + * m.doAssoc({'b':2}); + * }); */ - PersistentMap mapValues(f(V value)); + PMap withTransient(dynamic change(TMap)); /** * Returns a new map whose (key, value) pairs are the union of those of `this` @@ -123,8 +219,8 @@ abstract class PersistentMap implements Iterable> { * Note that [union] is commutative if and only if [combine] is provided and * if it is commutative. */ - PersistentMap - union(PersistentMap other, [V combine(V left, V right)]); + PMap + union(PMap other, [V combine(V left, V right)]); /** * Returns a new map whose (key, value) pairs are the intersection of those of @@ -143,71 +239,97 @@ abstract class PersistentMap implements Iterable> { * Note that [intersection] is commutative if and only if [combine] is * provided and if it is commutative. */ - PersistentMap - intersection(PersistentMap other, [V combine(V left, V right)]); - - /// Returns a mutable copy of `this`. - Map toMap(); - - /// The keys of `this`. - Iterable get keys; - - /// The values of `this`. - Iterable get values; - - /// Randomly picks an entry of `this`. - Pair pickRandomEntry([Random random]); + PMap + intersection(PMap other, [V combine(V left, V right)]); /// A strict (non-lazy) version of [map]. - PersistentMap strictMap(Pair f(Pair pair)); + PMap strictMap(Pair f(Pair pair)); /// A strict (non-lazy) version of [where]. - PersistentMap strictWhere(bool f(Pair pair)); + PMap strictWhere(bool f(Pair pair)); + + PMap update(K key, dynamic updateF); + } /** - * A base class for implementations of [PersistentMap]. + * A transient map, binding keys of type [K] to values of type [V]. Null values + * are supported but null keys are not. + * + * Transient data structure is a mutable structure, which can be efficiently + * converted to the persistent data structure. It is usually created from + * a persistent structure to apply some changes and obtain a new persistent + * structure. The less changes are done, the more efficient is the conversion. */ -abstract class PersistentMapBase - extends IterableBase> - implements PersistentMap { - - Map toMap() { - Map result = new Map(); - this.forEachKeyValue((K k, V v) { result[k] = v; }); - return result; - } - - String toString() { - StringBuffer buffer = new StringBuffer('{'); - bool comma = false; - this.forEachKeyValue((K k, V v) { - if (comma) buffer.write(', '); - buffer.write('$k: $v'); - comma = true; - }); - buffer.write('}'); - return buffer.toString(); - } +abstract class TMap implements ReadMap { - // Optimized version of Iterable's contains - bool contains(Pair entry) { - final value = this.lookup(entry.fst); - return value.isDefined && value.value == entry.snd; - } + /** + * Creates an empty map using the default implementation of + * [TMap]. + */ + factory TMap() => new _TMapImpl(); - Iterable get keys => this.map((Pair pair) => pair.fst); + /** + * Binds [key] to [value]. + * + * If [key] was bound to some `oldvalue`, it is nevertheless bound + * to [value]. + * + * var map = PersistentMap.fromMap({'a': 1}).asTransient(); + * map.doAssoc('b', 2); // map is now {'a': 1, 'b': 2} + * map.doAssoc('b', 3); // map is now {'a': 1, 'b': 3} + */ + TMap + doAssoc(K key, V value); - Iterable get values => this.map((Pair pair) => pair.snd); + /** + * Unbinds [key]. + * + * If [key] is not bound and [missingOk] is `false`, exception is thrown. + * If [key] is not bound and [missingOk] is specified as `true`, nothing happens. + * [missingOk] defaults to `false`. + * + * var map = PersistentMap.fromMap({'a': 1, 'b': 2}).asTransient(); + * map.doDelete('b', 2); // map is now {'a': 1} + * map.doDelete('b', 2, missingOk: true); // map is still {'a': 1} + */ + TMap doDelete(K key, {bool missingOk: false}) ; - Pair pickRandomEntry([Random random]) => - elementAt((random != null ? random : _random).nextInt(this.length)); + /** + * Adjusts the value that is possibly bound to [key] by applying [f]. + * + * [f] should have one of the following signatures: V f(V value), V f([V value]) + * + * If [key] is not bound, result of calling [f] with no arguments is associated with [key] + * If [key] is not bound and [f] can not be called with no arguments, [Exception] is thrown. + * + * var map = PersistentMap.fromMap({'a': 1}).asTransient(); + * map.doUpdate('b', (x) => x + 1); // throws + * map.doUpdate('b', ([x]) => x == null ? 2 : x + 1); // map is now {'a': 1, 'b': 2} + * map.doUpdate('b', (x) => x + 1); // map is now {'a': 1, 'b': 3} + */ + TMap doUpdate(K key, dynamic f); - V operator [](K key) => this.lookup(key).asNullable; + /** + * Updates all values by passing them to [f] and replacing them by results. + * + * var map = PersistentMap.fromMap({'a': 1, 'b': 2}).asTransient(); + * map.mapValues((x) => x + 1) // map is now {'a': 2, 'b': 3} + */ +// TMap doMapValues(f(V value)); - PersistentMap strictMap(Pair f(Pair pair)) => - new PersistentMap.fromPairs(this.map(f)); + /** + * Returns a persistent copy of `this`. + * + * This is ussualy called when changes to `this` + * are finished + * + * var persistent1 = new PersistentMap.from({'a':1}); + * var transient = persistent1.asTransient(); + * transient.doAssoc({'b':2}); + * var persistent2 = new transient.asPersistent(); + */ + PMap asPersistent(); - PersistentMap strictWhere(bool f(Pair pair)) => - new PersistentMap.fromPairs(this.where(f)); + operator []=(key, value); } diff --git a/lib/src/map_impl.dart b/lib/src/map_impl.dart index b5e6556..3552ca1 100644 --- a/lib/src/map_impl.dart +++ b/lib/src/map_impl.dart @@ -1,391 +1,616 @@ +// Copyright (c) 2014, VacuumLabs. // Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code // is governed by a BSD-style license that can be found in the LICENSE file. -// Author: Paul Brauner (polux@google.com) +// Authors are listed in the AUTHORS file part of persistent; + +// this turns out to be the fastest combination + +const branchingBits = 4; +const maxDepth = 6; + +//const branchingBits = 3; +//const maxDepth = 9; + +//const branchingBits = 5; +//const maxDepth = 5; + + +const branching = 1 << branchingBits; +const branchingMask = (1 << branchingBits)-1; +const allHashMask = (1 << (maxDepth+1) * branchingBits)-1; + +const leafSize = branching * 3; +const leafSizeMin = branching * 2; + +const binSearchThr = 4; +const recsize = 3; //0 - hash, 1 - key, 2 - val + final _random = new Random(); -/** - * Exception used for aborting forEach loops. - */ -class _Stop implements Exception {} +_ThrowKeyError(key) => throw new Exception('Key Error: ${key} is not defined'); -/** - * Superclass for _EmptyMap, _Leaf and _SubMap. - */ -abstract class _APersistentMap extends PersistentMapBase { - final int length; - - _APersistentMap(this.length, this.isEmpty, this._isLeaf); - - final bool isEmpty; - final bool _isLeaf; - - Option _lookup(K key, int hash, int depth); - PersistentMap _insertWith(LinkedList> keyValues, int size, - V combine(V x, V y), int hash, int depth); - PersistentMap _intersectWith(LinkedList> keyValues, int size, - V combine(V x, V y), int hash, int depth); - PersistentMap _delete(K key, int hash, int depth); - PersistentMap _adjust(K key, V update(V), int hash, int depth); - - _APersistentMap - _unionWith(_APersistentMap m, V combine(V x, V y), int depth); - _APersistentMap - _unionWithEmptyMap(_EmptyMap m, V combine(V x, V y), int depth); - _APersistentMap - _unionWithLeaf(_Leaf m, V combine(V x, V y), int depth); - _APersistentMap - _unionWithSubMap(_SubMap m, V combine(V x, V y), int depth); - - _APersistentMap - _intersectionWith(_APersistentMap m, V combine(V x, V y), - int depth); - _APersistentMap - _intersectionWithEmptyMap(_EmptyMap m, V combine(V x, V y), - int depth); - _APersistentMap - _intersectionWithLeaf(_Leaf m, V combine(V x, V y), int depth); - _APersistentMap - _intersectionWithSubMap(_SubMap m, V combine(V x, V y), int depth); - - Pair _elementAt(int index); - - LinkedList> _onePair(K key, V value) => - new Cons>(new Pair(key, value), new Nil>()); - - Option lookup(K key) => - _lookup(key, key.hashCode & 0x3fffffff, 0); - - PersistentMap insert(K key, V value, [V combine(V x, V y)]) => - _insertWith(_onePair(key, value), - 1, - (combine != null) ? combine : (V x, V y) => y, - key.hashCode & 0x3fffffff, 0); - - PersistentMap delete(K key) => - _delete(key, key.hashCode & 0x3fffffff, 0); - - PersistentMap adjust(K key, V update(V)) => - _adjust(key, update, key.hashCode & 0x3fffffff, 0); - - PersistentMap union(PersistentMap other, [V combine(V x, V y)]) => - this._unionWith(other, (combine != null) ? combine : (V x, V y) => y, 0); - - PersistentMap - intersection(PersistentMap other, [V combine(V left, V right)]) => - this._intersectionWith(other, - (combine != null) ? combine : (V x, V y) => y, 0); - - Pair elementAt(int index) { - if (index < 0 || index >= length) throw new RangeError.value(index); - return _elementAt(index); - } - - // toString() => toDebugString(); +_ThrowUpdateKeyError(key, exception) => throw new Exception('Key $key was not found, calling update with no arguments threw: $exception'); + +_getUpdateValue(key, updateF) { + try { + return updateF(); + } catch(e) { + _ThrowUpdateKeyError(key, e); + } } -class _EmptyMapIterator implements Iterator> { - const _EmptyMapIterator(); - Pair get current => null; - bool moveNext() => false; +/// see technical.md for explanation of what this does +_mangleHash(hash){ + var _tmp = hash ^ (hash << 8); + return ((_tmp ^ (_tmp << 16)) & allHashMask); +} + +/// no array compression here (aka bitpos style). Why? See technical.md + +_getBranch(hash, depth){ + return (hash >> (depth*branchingBits)) & branchingMask; } -class _EmptyMap extends _APersistentMap { - _EmptyMap() : super(0, true, false); - Option _lookup(K key, int hash, int depth) => new Option.none(); - PersistentMap _insertWith( - LinkedList> keyValues, int size, V combine(V x, V y), int hash, - int depth) { - assert(size == keyValues.length); - return new _Leaf(hash, keyValues, size); +class _TMapImpl + extends IterableBase> + implements TMap { + // Although PMap can be represented a simple _Node, Transient map needs + // separate structure with a mutable reference to _Node. + _Node _root; + + _Owner _owner; + get owner => _owner != null ? _owner : + throw new Exception('Cannot modify TMap after calling asPersistent.'); + + factory _TMapImpl() => new _TMapImpl.fromPersistent(new PMap()); + + /** + * Creates an immutable copy of [map] using the default implementation of + * [TMap]. + */ + _TMapImpl.fromPersistent(_Node map) { + _owner = new _Owner(); + _root = map; } - PersistentMap _intersectWith(LinkedList> keyValues, int size, - V combine(V x, V y), int hash, int depth) { - assert(size == keyValues.length); + TMap _adjustRootAndReturn(newRoot) { + _root = newRoot; return this; } - PersistentMap _delete(K key, int hash, int depth) => this; + TMap + doAssoc(K key, V value) { + return _adjustRootAndReturn(_root._assoc(owner, key, value)); + } - PersistentMap _adjust(K key, V update(V), int hash, int depth) => this; + operator []=(key, value){ + this.doAssoc(key, value); + } - PersistentMap - _unionWith(PersistentMap m, V combine(V x, V y), int depth) => m; + TMap doDelete(K key, {bool missingOk: false}) { + return _adjustRootAndReturn(_root._delete(owner, key, _mangleHash(key.hashCode), maxDepth, missingOk)); + } - PersistentMap - _unionWithEmptyMap(_EmptyMap m, V combine(V x, V y), int depth) { - throw "should never be called"; + TMap doUpdate(K key, dynamic updateF) { + return _adjustRootAndReturn(_root._update(owner, key, updateF)); } - PersistentMap - _unionWithLeaf(_Leaf m, V combine(V x, V y), int depth) => m; + PMap asPersistent() { + _owner = null; + return this._root; + } - PersistentMap - _unionWithSubMap(_SubMap m, V combine(V x, V y), int depth) => m; + toString() => 'TMap($_root)'; - PersistentMap - _intersectionWith(_APersistentMap m, V combine(V x, V y), - int depth) => this; + V get(K key, [V notFound = _none]) => _root.get(key, notFound); - PersistentMap - _intersectionWithEmptyMap(_EmptyMap m, V combine(V x, V y), - int depth) { - throw "should never be called"; - } + V operator [](K key) => _root.get(key); - PersistentMap _intersectionWithLeaf( - _Leaf m, V combine(V x, V y), int depth) => this; + void forEachKeyValue(f) => _root.forEachKeyValue(f); - PersistentMap _intersectionWithSubMap( - _SubMap m, V combine(V x, V y), int depth) => this; + Map toMap() =>_root.toMap(); - PersistentMap mapValues(f(V)) => this; + Iterable get keys => _root.keys; - void forEachKeyValue(f(K, V)) {} + Iterable get values => _root.values; - bool operator ==(PersistentMap other) => other is _EmptyMap; + Iterator get iterator => _root.iterator; - Iterator> get iterator => const _EmptyMapIterator(); + int get length => _root.length; - Pair _elementAt(int index) { - throw new RangeError.value(index); - } + bool containsKey(key) => _root.containsKey(key); - Pair get last { - throw new StateError("Empty map has no entries"); - } + bool hasKey(key) => _root.hasKey(key); - toDebugString() => "_EmptyMap()"; } -class _Leaf extends _APersistentMap { + +/** + * Superclass for _EmptyMap, _Leaf and _SubMap. + */ +abstract class _Node extends IterableBase> implements PMap { + _Owner _owner; + int _length; int _hash; - LinkedList> _pairs; - - _Leaf(this._hash, pairs, int size) : super(size, false, true) { - this._pairs = pairs; - assert(size == pairs.length); - } - - PersistentMap _insertWith(LinkedList> keyValues, int size, - V combine(V x, V y), int hash, int depth) { - assert(size == keyValues.length); - // newsize is incremented as a side effect of insertPair - int newsize = length; - - LinkedList> insertPair(Pair toInsert, - LinkedList> pairs) { - LinkedListBuilder> builder = - new LinkedListBuilder>(); - LinkedList> it = pairs; - while (it.isCons) { - Cons> cons = it.asCons; - Pair elem = cons.elem; - if (elem.fst == toInsert.fst) { - builder.add(new Pair( - toInsert.fst, - combine(elem.snd, toInsert.snd))); - return builder.build(cons.tail); + get length => _length; + + _Node(this._owner, this._length); + + factory _Node.fromMap(map){ + _Node root = new _Leaf.empty(null); + map.forEach((K key, V value) { + root = root._assoc(null, key, value); + }); + return root; + } + + factory _Node.fromPairs(pairs){ + var _root = new _Leaf.empty(null); + pairs.forEach((pair) { + _root = _root._assoc(null, pair.fst, pair.snd); + }); + return _root; + } + + _forEachKVSegment(f); + + V _get(K key, int hash, int depth); + _Node _insertOneWith(_Owner owner, key, val, hash, int depth, [update]); + + int get hashCode; + + _Node _update(_Owner owner, K key, dynamic updateF){ + return _insertOneWith(owner, key, null, _mangleHash(key.hashCode), maxDepth, updateF); + } + + PMap update(K key, dynamic updateF) => + _insertOneWith(null, key, null, _mangleHash(key.hashCode), maxDepth, updateF); + + _Node _assoc(_Owner owner, K key, V value) => + _insertOneWith(owner, key, value, _mangleHash(key.hashCode), maxDepth); + + PMap assoc(K key, V value) => _assoc(null, key, value); + + _Node _delete(_Owner owner, K key, int hash, int depth, bool missingOk); + + PMap delete(K key, {bool missingOk: false}) => _delete(null, key, _mangleHash(key.hashCode), maxDepth, missingOk); + + bool operator ==(other) { + if (other is! _Node) return false; + if (identical(this, other)) return true; + _Node me = this; + if (me.length != other.length) { + return false; + } + if (me is _Leaf && other is _Leaf) { + List mekv = (me as _Leaf)._kv; + List okv = other._kv; + var lastMatch=0; + for (int i=0; i> insertPairs( - LinkedList> toInsert, LinkedList> pairs) { - LinkedList> res = pairs; - LinkedList> it = toInsert; - while (it.isCons) { - Cons> cons = it.asCons; - Pair elem = cons.elem; - res = insertPair(elem, res); - it = cons.tail; + return true; + } + if (me is _SubMap && other is _SubMap) { + for (int i=0; i 5) { - assert(_hash == hash); - final LinkedList> newPairs = insertPairs(keyValues, _pairs); - return new _Leaf(hash, newPairs, newsize); - } else { - if (hash == _hash) { - final LinkedList> newPairs = insertPairs(keyValues, _pairs); - return new _Leaf(hash, newPairs, newsize); + // method must be called only on top-level _Node + V get(K key, [V notFound = _none]) { + var val = _get(key, _mangleHash(key.hashCode), maxDepth); + if(_isNone(val)){ + if (_isNone(notFound)) { + _ThrowKeyError(key); + return null; } else { - int branch = (_hash >> (depth * 5)) & 0x1f; - List<_APersistentMap> array = <_APersistentMap>[this]; - return new _SubMap(1 << branch, array, length) - ._insertWith(keyValues, size, combine, hash, depth); + return notFound; } + } else { + return val; } } - PersistentMap _intersectWith(LinkedList> keyValues, int size, - V combine(V x, V y), int hash, int depth) { - assert(size == keyValues.length); - // TODO(polux): possibly faster implementation - Map map = toMap(); - LinkedListBuilder> builder = new LinkedListBuilder>(); - int newsize = 0; - keyValues.foreach((Pair pair) { - if (map.containsKey(pair.fst)) { - builder.add(new Pair(pair.fst, combine(map[pair.fst], pair.snd))); - newsize++; - } - }); - return new _Leaf(_hash, builder.build(), newsize); + Map toMap() { + Map result = new Map(); + this.forEachKeyValue((K k, V v) { result[k] = v; }); + return result; } - PersistentMap _delete(K key, int hash, int depth) { - if (hash != _hash) - return this; - bool found = false; - LinkedList> newPairs = _pairs.strictWhere((p) { - if (p.fst == key) { - found = true; - return false; - } - return true; + String toString() { + StringBuffer buffer = new StringBuffer('{'); + bool comma = false; + this.forEachKeyValue((K k, V v) { + if (comma) buffer.write(', '); + buffer.write('$k: $v'); + comma = true; }); - return newPairs.isNil - ? new _EmptyMap() - : new _Leaf(_hash, newPairs, found ? length - 1 : length); - } - - PersistentMap _adjust(K key, V update(V), int hash, int depth) { - LinkedList> adjustPairs() { - LinkedListBuilder> builder = - new LinkedListBuilder>(); - LinkedList> it = _pairs; - while (it.isCons) { - Cons> cons = it.asCons; - Pair elem = cons.elem; - if (elem.fst == key) { - builder.add(new Pair(key, update(elem.snd))); - return builder.build(cons.tail); + buffer.write('}'); + return buffer.toString(); + } + + Iterable get keys => this.map((Pair pair) => pair.fst); + + Iterable get values => this.map((Pair pair) => pair.snd); + + void forEachKeyValue(f(K key, V value)); + + bool containsKey(key) { + final _none = new Object(); + final value = this.get(key, _none); + return value != _none; + } + + bool hasKey(key) => containsKey(key); + + TMap asTransient() { + return new _TMapImpl.fromPersistent(this); + } + + PMap withTransient(dynamic f(TMap map)) { + TMap transient = this.asTransient(); + f(transient); + return transient.asPersistent(); + } + + V operator [](K key) => get(key); + + PMap strictMap(Pair f(Pair pair)) => + new PMap.fromPairs(this.map(f)); + + PMap strictWhere(bool f(Pair pair)) => + new PMap.fromPairs(this.where(f)); + + static _returnRight(left, right) => right; + + PMap union( + PMap other, + [V combine(V left, V right) = _returnRight] + ){ + if(other is _Node){ + return _union(other as _Node, combine, maxDepth); + } else { + var result = this.asTransient(); + other.forEachKeyValue((K key, V value){ + if(result.hasKey(key)){ + result.doAssoc(key, combine(this[key], value)); + } else { + result.doAssoc(key, value); } - builder.add(elem); - it = cons.tail; - } - return builder.build(); + }); + return result.asPersistent(); } + } - return (hash != _hash) - ? this - : new _Leaf(_hash, adjustPairs(), length); + PMap intersection( + PMap other, + [V combine(V left, V right) = _returnRight] + ){ + if(other is _Node){ + return _intersection(other as _Node, combine, maxDepth); + } else { + var result = new PMap().asTransient(); + other.forEachKeyValue((K key, V value){ + if(this.hasKey(key)){ + result.doAssoc(key, combine(this[key], value)); + } + }); + return result.asPersistent(); + } } - PersistentMap - _unionWith(_APersistentMap m, V combine(V x, V y), int depth) => - m._unionWithLeaf(this, combine, depth); + PMap _union( + _Node other, + V combine(V left, V right), + int depth + ); + + PMap _intersection( + _Node other, + V combine(V left, V right), + int depth + ); +} - PersistentMap - _unionWithEmptyMap(_EmptyMap m, V combine(V x, V y), int depth) => - this; - PersistentMap - _unionWithLeaf(_Leaf m, V combine(V x, V y), int depth) => - m._insertWith(_pairs, length, combine, _hash, depth); +class _Leaf extends _Node { + List _kv; + get private_kv => _kv; - PersistentMap - _unionWithSubMap(_SubMap m, V combine(V x, V y), int depth) => - m._insertWith(_pairs, length, combine, _hash, depth); + get iterator { + List> pairs = []; + for (int i=0; i<_kv.length; i+=recsize){ + pairs.add(new Pair(_kv[i+1], _kv[i+2])); + } + return pairs.iterator; + } - PersistentMap _intersectionWith(_APersistentMap m, - V combine(V x, V y), int depth) => - m._intersectionWithLeaf(this, combine, depth); + /// check whether order-by-hashcode invariant (see technical.md) holds + void sanityCheck(){ + var lasthash = double.NEGATIVE_INFINITY; + for (int i=0; i<_kv.length; i+=recsize){ + if (lasthash>_kv[i]) { + throw new Exception('invariant violated'); + } + lasthash = _kv[i]; + } + } - PersistentMap _intersectionWithEmptyMap(_EmptyMap m, - V combine(V x, V y), - int depth) => - m; + int get hashCode { + if(_hash != null) return _hash; + _hash = 0; + for(int i=0; i<_kv.length; i+=recsize){ + _hash ^= hash2(_kv[i], _kv[i+2].hashCode); - PersistentMap _intersectionWithLeaf(_Leaf m, V combine(V x, V y), - int depth) => - m._intersectWith(_pairs, length, combine, _hash, depth); + } + return _hash; + } - PersistentMap _intersectionWithSubMap(_SubMap m, - V combine(V x, V y), int depth) => - m._intersectWith(_pairs, length, combine, _hash, depth); + _Leaf.abc(_Owner owner, _kv) : super(owner, _kv.length ~/ recsize){ + this._kv = _kv; + } + + _Leaf.empty(_Owner owner): super(owner, 0) { + this._kv = []; + } - Option _lookup(K key, int hash, int depth) { - if (hash != _hash) - return new Option.none(); - LinkedList> it = _pairs; - while (it.isCons) { - Cons> cons = it.asCons; - Pair elem = cons.elem; - if (elem.fst == key) return new Option.some(elem.snd); - it = cons.tail; + factory _Leaf.fromSubmap(_Owner owner, _SubMap sm) { + List _kv = []; + sm._forEachKVSegment((kv){ + _kv.addAll(kv); + }); + var nres = new _Leaf.abc(owner, _kv); + return nres; + } + + factory _Leaf.ensureOwner(_Leaf old, _Owner owner, kv, int length) { + if(_ownerEquals(owner, old._owner)) { + old._kv = kv; + old._length = length; + return old; } - return new Option.none(); + return new _Leaf.abc(owner, kv); } - PersistentMap mapValues(f(V)) => - new _Leaf(_hash, - _pairs.strictMap((p) => new Pair(p.fst, f(p.snd))), length); + /// see technical.md for explanation of what this does - void forEachKeyValue(f(K, V)) { - _pairs.foreach((Pair pair) => f(pair.fst, pair.snd)); + _Node _polish(_Owner owner, int depth, List _kv) { + assert(_kv.length % recsize == 0); + // depth == -1 means we are at the bottom level; we consumed all + // information from 'hash' and we have to extend the _Leaf no matter how + // long it gets + if (_kv.length < recsize * leafSize || depth == -1) { + return new _Leaf.abc(owner, _kv); + } else { + List kvs = new List.generate(branching, (_) => []); + for (int i=0; i<_kv.length; i+=recsize){ + int branch = _getBranch(_kv[i], depth); + kvs[branch].add(_kv[i]); + kvs[branch].add(_kv[i + 1]); + kvs[branch].add(_kv[i + 2]); + } + List <_Node> array = new List.generate(branching, + (i) => new _Leaf.abc(owner, kvs[i])); + return new _SubMap.abc(owner, array, _kv.length ~/ recsize); + } } - bool operator ==(PersistentMap other) { - if (identical(this, other)) return true; - if (other is! _Leaf) return false; - _Leaf otherLeaf = other; - if (_hash != otherLeaf._hash) return false; - if (length != otherLeaf.length) return false; - Map thisAsMap = toMap(); - int counter = 0; - LinkedList> it = otherLeaf._pairs; - while (it.isCons) { - Cons> cons = it.asCons; - Pair elem = cons.elem; - if (elem.snd == null && !thisAsMap.containsKey(elem.fst)) - return false; - if (thisAsMap[elem.fst] != elem.snd) - return false; - counter++; - it = cons.tail; + _insert(List into, key, val, hash, [update]){ + assert(into.length % recsize == 0); + if (into.length == 0) { + into.addAll([hash, key, val]); + return; + } + int from = 0; + int to = (into.length ~/ recsize) - 1; + while(to - from > binSearchThr){ + int mid = (from + to) ~/ 2; + var midh = into[mid*recsize]; + if (midh > hash){ + to = mid; + } else if (midh < hash){ + from = mid; + } else { + break; + } + } + + for (int i=from*recsize; i<=to*recsize; i+=recsize) { + assert(i%recsize == 0); + if (hash <= into[i]) { + if (hash < into[i]) { + into.insertAll(i, [hash, key, val]); + return; + } + if (key == into[i+1]) { + if (update == null) { + into[i+2] = val; + } else { + into[i+2] = update(into[i+2]); + } + return; + } + } } - return thisAsMap.length == counter; + + if (update == null) { + into.addAll([hash, key, val]); + } else { + into.addAll([hash, key, _getUpdateValue(key, update)]); + } + assert(into.length % recsize == 0); + } + + _Node _insertOneWith(_Owner owner, key, val, hash, int depth, [update]) { + List nkv = _makeCopyIfNeeded(owner, this._owner, _kv); + _insert(nkv, key, val, hash, update); + return _polish(owner, depth, nkv); } - Iterator> get iterator => _pairs.iterator; + _Node _delete(_Owner owner, K key, int hash, int depth, bool missingOk) { + bool found = false; + List nkv = _makeCopyIfNeeded(owner, this._owner, _kv); + for (int i=0; i.ensureOwner(this, owner, nkv, nkv.length ~/ recsize); + } + } - Pair _elementAt(int index) { - var tail = _pairs; - for (int i = 0; i < index; i++) { - tail = tail.asCons.tail; + V _get(K key, int hash, int depth) { + int f=0; + int from = 0; + int to = _kv.length ~/ recsize; + while(to - from > binSearchThr){ + int mid = (from + to) ~/ 2; + var midh = _kv[mid * recsize]; + if (midh > hash){ + to = mid; + } else if (midh < hash){ + from = mid; + } else { + break; + } + } + for(int i=from*recsize; i get last { - Cons pairs = _pairs.asCons; - while (!pairs.tail.isNil) { - pairs = pairs.tail; + void forEachKeyValue(f(K, V)) { + for (int i=0; i<_kv.length; i+=recsize) { + f(_kv[i+1], _kv[i+2]); } - return pairs.elem; } - toDebugString() => "_Leaf($_hash, $_pairs)"; + toDebugString() => "_Leaf($_kv)"; + + _forEachKVSegment(f){ + f(_kv); + } + + PMap _union( + _Node other, + V combine(V left, V right), + int depth + ){ + // TODO: More efficient union of two leafs. + _Owner owner = new _Owner(); + for(int i = 0; i < _kv.length; i+=3){ + var res = other._get(_kv[i+1], _kv[i], depth); + if(_isNone(res)){ + other = + other._insertOneWith(owner, _kv[i+1], _kv[i+2], _kv[i], depth); + } else { + other = + other._insertOneWith( + owner, _kv[i+1], + combine(_kv[i+2], res), + _kv[i], + depth + ); + } + } + other._owner = null; + return other; + } + + PMap _intersection( + _Node other, + V combine(V left, V right), + int depth + ){ + List _nkv = []; + for(int i = 0; i < _kv.length; i+=3){ + var res = other._get(_kv[i+1], _kv[i], depth); + if(!_isNone(res)){ + _nkv.addAll([_kv[i], _kv[i+1], combine(_kv[i+2], res)]); + } + } + return new _Leaf.abc(null, _nkv); + } } class _SubMapIterator implements Iterator> { - List<_APersistentMap> _array; + List<_Node> _array; int _index = 0; // invariant: _currentIterator != null => _currentIterator.current != null Iterator> _currentIterator = null; @@ -411,295 +636,137 @@ class _SubMapIterator implements Iterator> { } } -class _SubMap extends _APersistentMap { - int _bitmap; - List<_APersistentMap> _array; - _SubMap(this._bitmap, this._array, int size) : super(size, false, false); - - static _popcount(int n) { - n = n - ((n >> 1) & 0x55555555); - n = (n & 0x33333333) + ((n >> 2) & 0x33333333); - n = (n + (n >> 4)) & 0x0F0F0F0F; - n = n + (n >> 8); - n = n + (n >> 16); - return n & 0x0000003F; - } - - Option _lookup(K key, int hash, int depth) { - int branch = (hash >> (depth * 5)) & 0x1f; - int mask = 1 << branch; - if ((_bitmap & mask) != 0) { - int index = _popcount(_bitmap & (mask - 1)); - _APersistentMap map = _array[index]; - return map._lookup(key, hash, depth + 1); - } else { - return new Option.none(); - } - } +class _SubMap extends _Node { + List<_Node> _array; - PersistentMap _insertWith(LinkedList> keyValues, int size, - V combine(V x, V y), int hash, int depth) { - assert(size == keyValues.length); + Iterator> get iterator => new _SubMapIterator(_array); - int branch = (hash >> (depth * 5)) & 0x1f; - int mask = 1 << branch; - int index = _popcount(_bitmap & (mask - 1)); + _SubMap.abc(_Owner owner, this._array, int length) : super(owner, length); - if ((_bitmap & mask) != 0) { - List<_APersistentMap> newarray = - new List<_APersistentMap>.from(_array, growable: false); - _APersistentMap m = _array[index]; - _APersistentMap newM = - m._insertWith(keyValues, size, combine, hash, depth + 1); - newarray[index] = newM; - int delta = newM.length - m.length; - return new _SubMap(_bitmap, newarray, length + delta); - } else { - int newlength = _array.length + 1; - List<_APersistentMap> newarray = - new List<_APersistentMap>(newlength); - // TODO: find out if there's a "copy array" native function somewhere - for (int i = 0; i < index; i++) { newarray[i] = _array[i]; } - for (int i = index; i < newlength - 1; i++) { newarray[i+1] = _array[i]; } - newarray[index] = new _Leaf(hash, keyValues, size); - return new _SubMap(_bitmap | mask, newarray, length + size); + factory _SubMap.ensureOwner(_SubMap old, _Owner owner, array, int length) { + if(_ownerEquals(owner, old._owner)) { + old._array = array; + old._length = length; } + return new _SubMap.abc(owner, array, length); } - PersistentMap _intersectWith(LinkedList> keyValues, int size, - V combine(V x, V y), int hash, int depth) { - assert(size == keyValues.length); - - int branch = (hash >> (depth * 5)) & 0x1f; - int mask = 1 << branch; + int get hashCode { + if(_hash != null) return _hash; + _hash = 0; + for(var child in _array){ + _hash ^= child.hashCode; - if ((_bitmap & mask) != 0) { - int index = _popcount(_bitmap & (mask - 1)); - _APersistentMap m = _array[index]; - return m._intersectWith(keyValues, size, combine, hash, depth + 1); - } else { - return new _EmptyMap(); } + return _hash; } - PersistentMap _delete(K key, int hash, int depth) { - int branch = (hash >> (depth * 5)) & 0x1f; - int mask = 1 << branch; + V _get(K key, int hash, int depth) { + int branch = _getBranch(hash, depth); + _Node map = _array[branch]; + return map._get(key, hash, depth - 1); + } - if ((_bitmap & mask) != 0) { - int index = _popcount(_bitmap & (mask - 1)); - _APersistentMap m = _array[index]; - _APersistentMap newm = m._delete(key, hash, depth + 1); - int delta = newm.length - m.length; - if (identical(m, newm)) { - return this; - } - if (newm.isEmpty) { - if (_array.length > 2) { - int newsize = _array.length - 1; - List<_APersistentMap> newarray = - new List<_APersistentMap>(newsize); - for (int i = 0; i < index; i++) { newarray[i] = _array[i]; } - for (int i = index; i < newsize; i++) { newarray[i] = _array[i + 1]; } - assert(newarray.length >= 2); - return new _SubMap(_bitmap ^ mask, newarray, length + delta); - } else { - assert(_array.length == 2); - assert(index == 0 || index == 1); - _APersistentMap onlyValueLeft = _array[1 - index]; - return onlyValueLeft._isLeaf - ? onlyValueLeft - : new _SubMap(_bitmap ^ mask, - <_APersistentMap>[onlyValueLeft], - length + delta); - } - } else if (newm._isLeaf){ - if (_array.length == 1) { - return newm; - } else { - List<_APersistentMap> newarray = - new List<_APersistentMap>.from(_array, growable: false); - newarray[index] = newm; - return new _SubMap(_bitmap, newarray, length + delta); - } - } else { - List<_APersistentMap> newarray = - new List<_APersistentMap>.from(_array, growable: false); - newarray[index] = newm; - return new _SubMap(_bitmap, newarray, length + delta); - } - } else { + _Node _insertOneWith(_Owner owner, key, val, hash, int depth, [update]) { + int branch = _getBranch(hash, depth); + _Node m = _array[branch]; + int oldSize = m.length; + _Node newM = m._insertOneWith(owner, key, val, hash, depth - 1, update); + if(identical(m, newM)) { + if(oldSize != m.length) this._length += m.length - oldSize; return this; } + List<_Node> newarray = _makeCopyIfNeeded(owner, this._owner, _array); + newarray[branch] = newM; + int delta = newM.length - oldSize; + return new _SubMap.ensureOwner(this, owner, newarray, length + delta); } - PersistentMap _adjust(K key, V update(V), int hash, int depth) { - int branch = (hash >> (depth * 5)) & 0x1f; - int mask = 1 << branch; - if ((_bitmap & mask) != 0) { - int index = _popcount(_bitmap & (mask - 1)); - _APersistentMap m = _array[index]; - _APersistentMap newm = m._adjust(key, update, hash, depth + 1); - if (identical(newm, m)) { - return this; - } - List<_APersistentMap> newarray = - new List<_APersistentMap>.from(_array, growable: false); - newarray[index] = newm; - return new _SubMap(_bitmap, newarray, length); - } else { + _Node _delete(owner, K key, int hash, int depth, bool missingOk) { + int branch = _getBranch(hash, depth); + _Node child = _array[branch]; + int childLength = child.length; + // need to remember child length as this may modify + // the child (if working with transient) + _Node newChild = child._delete(owner, key, hash, depth - 1, missingOk); + int newLength = this.length + (newChild.length - childLength); + if (identical(child, newChild)) { + this._length = newLength; return this; } - } + List<_Node> newarray = new List<_Node>.from(_array); + newarray[branch] = newChild; + _Node res = new _SubMap.ensureOwner(this, owner, newarray, newLength); - PersistentMap - _unionWith(_APersistentMap m, V combine(V x, V y), int depth) => - m._unionWithSubMap(this, combine, depth); - - PersistentMap - _unionWithEmptyMap(_EmptyMap m, V combine(V x, V y), int depth) => - this; - - PersistentMap - _unionWithLeaf(_Leaf m, V combine(V x, V y), int depth) => - this._insertWith(m._pairs, m.length, (V v1, V v2) => combine(v2, v1), - m._hash, depth); - - PersistentMap - _unionWithSubMap(_SubMap m, V combine(V x, V y), int depth) { - int ormap = _bitmap | m._bitmap; - int andmap = _bitmap & m._bitmap; - List<_APersistentMap> newarray = - new List<_APersistentMap>(_popcount(ormap)); - int mask = 1, i = 0, i1 = 0, i2 = 0; - int newSize = 0; - while (mask <= ormap) { - if ((andmap & mask) != 0) { - _array[i1]; - m._array[i2]; - _APersistentMap newMap = - m._array[i2]._unionWith(_array[i1], combine, depth + 1); - newarray[i] = newMap; - newSize += newMap.length; - i1++; - i2++; - i++; - } else if ((_bitmap & mask) != 0) { - _APersistentMap newMap = _array[i1]; - newarray[i] = newMap; - newSize += newMap.length; - i1++; - i++; - } else if ((m._bitmap & mask) != 0) { - _APersistentMap newMap = m._array[i2]; - newarray[i] = newMap; - newSize += newMap.length; - i2++; - i++; - } - mask <<= 1; - } - return new _SubMap(ormap, newarray, newSize); - } - - PersistentMap _intersectionWith(_APersistentMap m, - V combine(V x, V y), int depth) => - m._intersectionWithSubMap(this, combine, depth); - - PersistentMap _intersectionWithEmptyMap(_EmptyMap m, - V combine(V x, V y), - int depth) => - m; - - PersistentMap _intersectionWithLeaf(_Leaf m, V combine(V x, V y), - int depth) => - _intersectWith(m._pairs, m.length, (V v1, V v2) => combine(v2, v1), - m._hash, depth); - - PersistentMap _intersectionWithSubMap( - _SubMap m, V combine(V x, V y), int depth) { - int andmap = _bitmap & m._bitmap; - List<_APersistentMap> newarray = new List<_APersistentMap>(); - int mask = 1, i1 = 0, i2 = 0; - int newSize = 0; - int newMask = 0; - while (mask <= _bitmap) { - if ((andmap & mask) != 0) { - _array[i1]; - m._array[i2]; - _APersistentMap newMap = - m._array[i2]._intersectionWith(_array[i1], combine, depth + 1); - newarray.add(newMap); - newSize += newMap.length; - newMask |= mask; - i1++; - i2++; - } else if ((_bitmap & mask) != 0) { - i1++; - } else if ((m._bitmap & mask) != 0) { - i2++; - } - mask <<= 1; - } - if (newarray.length > 1) { - return new _SubMap(newMask, newarray, newSize); - } else if (newarray.length == 1) { - _APersistentMap onlyValueLeft = newarray[0]; - return onlyValueLeft._isLeaf - ? onlyValueLeft - : new _SubMap(newMask, newarray, newSize); + // if submap is too small, let's replace it by _Leaf + if (res._length >= leafSizeMin) { + return res; } else { - return new _EmptyMap(); + return new _Leaf.fromSubmap(owner, res); } } - PersistentMap mapValues(f(V)) { - List<_APersistentMap> newarray = - new List<_APersistentMap>.from(_array, growable: false); - for (int i = 0; i < _array.length; i++) { - _APersistentMap mi = _array[i]; - newarray[i] = mi.mapValues(f); - } - return new _SubMap(_bitmap, newarray, length); + _forEachKVSegment(f) { + _array.forEach((child) => child._forEachKVSegment(f)); } forEachKeyValue(f(K, V)) { _array.forEach((mi) => mi.forEachKeyValue(f)); } - bool operator ==(PersistentMap other) { - if (identical(this, other)) return true; - if (other is! _SubMap) return false; - _SubMap otherSubMap = other; - if (_bitmap != otherSubMap._bitmap) return false; - if (length != otherSubMap.length) return false; - assert(_array.length == otherSubMap._array.length); - for (int i = 0; i < _array.length; i++) { - _APersistentMap mi = _array[i]; - _APersistentMap omi = otherSubMap._array[i]; - if (mi != omi) { - return false; - } + toDebugString() => "_SubMap($_array)"; + + PMap _union( + _Node other, + V combine(V left, V right), + int depth + ){ + if(other is _SubMap){ + var children = new List.generate(branching, (int i) => ( + _array[i]._union((other as _SubMap)._array[i], combine, depth-1) + )); + int size = children.fold(0, (int sum, _Node x) => + sum+=x.length + ); + return new _SubMap.abc(null, children, size); + } else { + return other._union(this, (x,y) => combine(y,x), depth); } - return true; } - Iterator> get iterator => new _SubMapIterator(_array); - - Pair get last => _array.last.last; - - Pair _elementAt(int index) { - int newIndex = index; - for (final subMap in _array) { - int subLength = subMap.length; - if (newIndex < subLength) { - return subMap._elementAt(newIndex); + PMap _intersection( + _Node other, + V combine(V left, V right), + int depth + ){ + if(other is _SubMap){ + var children = new List.generate(branching, (int i) => ( + _array[i]._intersection((other as _SubMap)._array[i], combine, depth-1) + )); + int size = children.fold(0, (int sum, _Node x) => + sum+=x.length + ); + var res = new _SubMap.abc(null, children, size); + if (size >= leafSizeMin) { + return res; + } else { + return new _Leaf.fromSubmap(null, res); } - newIndex -= subLength; + } else { + return other._intersection(this, (x,y) => combine(y,x), depth); } - throw new StateError("never happens"); } +} - toDebugString() => "_SubMap($_array)"; +_ownerEquals(_Owner a, _Owner b) { + return a != null && a == b; +} + +/// usually, we need to copy some arrays when associng. However, when working +/// with transients (and the owners match), it is safe just to modify the array +_makeCopyIfNeeded(_Owner a, _Owner b, List c) { + if(_ownerEquals(a, b)) + return c; + else return c.sublist(0); } diff --git a/lib/src/option.dart b/lib/src/option.dart deleted file mode 100644 index 245b019..0000000 --- a/lib/src/option.dart +++ /dev/null @@ -1,57 +0,0 @@ -// Copyright (c) 2012, Google Inc. All rights reserved. Use of this source code -// is governed by a BSD-style license that can be found in the LICENSE file. - -// Author: Paul Brauner (polux@google.com) - -part of persistent; - -class Option { - static final _none = new Option._internal(false, null); - - final T _value; - final bool isDefined; - - Option._internal(this.isDefined, this._value); - - factory Option.none() => _none; - - factory Option.some(T value) => new Option._internal(true, value); - - factory Option.fromNullable(T nullableValue) => - nullableValue == null ? _none : new Option.some(nullableValue); - - T get value { - if (isDefined) return _value; - throw new StateError('Option.none() has no value'); - } - - T get asNullable => isDefined ? _value : null; - - T orElse(T defaultValue) => isDefined ? _value : defaultValue; - - T orElseCompute(T defaultValue()) => isDefined ? _value : defaultValue(); - - /// [:forall U, Option map(U f(T value)):] - Option map(f(T value)) => - isDefined ? new Option.some(f(_value)) : this; - - /// [:forall U, Option map(Option f(T value)):] - Option expand(Option f(T value)) => - isDefined ? f(_value) : this; - - /// Precondition: [:this is Option