Skip to content

Big POMDP update #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
TheGreatfpmK wants to merge 68 commits into
base: master
Choose a base branch
from
Draft

Big POMDP update #39

TheGreatfpmK wants to merge 68 commits into from

Conversation

@TheGreatfpmK
Copy link
Collaborator

@TheGreatfpmK TheGreatfpmK commented Apr 19, 2024 •
edited
Loading

Currently requires:

Summary of changes:

  • Added New SAYNT loop and its related features
  • Added alpha vector parsing and analysis
  • Added support for native discounting (added example property file for discounted specification)
  • Added models highlighting problems of SAYNT and SARSOP (used in CAV SI paper)
  • Improved export into Cassandra POMDP format, mainly for Pmax and Rmin properties
  • Added more options for Storm POMDP belief explorer

!!! Don't merge until I finish some QoL changes and until Storm/StormPy is also merged !!!

Still work in progress:

  • Add few CLI options that are more user friendly (--saynt, --new-saynt etc.)
  • Code quality in some files

Waiting for merge:

  • I will add dependence on Storm/StormPy pull requests here once they are up
  • Without the Storm merge it's not even possible to build Payntbind because of some references!

@TheGreatfpmK
initial belief exploration implementation
4c11342
@TheGreatfpmK
Merge branch 'master' into belief-exploration
683b77d
@TheGreatfpmK
Prepared for experiments
dbc126a
@TheGreatfpmK
belief experiments
c6a4785
@TheGreatfpmK
use Storm results to help the synthesis
10df8d4
@TheGreatfpmK
Added average observation belief computation
881c5ce
@TheGreatfpmK
Iterative belief FSC synthesis
45cc3c1
@TheGreatfpmK
First version of enhanced SAYNT (add logging)
f907d4b
@TheGreatfpmK
Main enhanced SAYNT loop done
ebb3adc
@TheGreatfpmK
Finished enhanced SAYNT
4dec063
@TheGreatfpmK
first experiments for enhanced SAYNT
6011564
@TheGreatfpmK
More experiments
6bb8bb5
@TheGreatfpmK
More experiments
dcdf6cf
@TheGreatfpmK
enhanced SAYNT Storm first implementation
3182bc6
@TheGreatfpmK
Removed duplicit experiments
cfc1fbc
@TheGreatfpmK
new order experiments
5315da2
@TheGreatfpmK
added SAYNT logs and graph generation
185a2b1
@TheGreatfpmK
more experiments
ff9aae5
@TheGreatfpmK
Updated graph generation
f523d92
@TheGreatfpmK
more experiments
d33beda
@TheGreatfpmK
Discounted value iteration initial support
24a7de7
@TheGreatfpmK
Changed how infinity EVTs are treated for maximizing properties
75c7e77
@TheGreatfpmK
Merge remote-tracking branch 'upstream/master' into belief-exploration
e827006
@TheGreatfpmK
Added support for computing discounted expected visits
4440130
@TheGreatfpmK
Changed enhanced SAYNT threads list for better code readability; Over…
41947b4 
…-approximations from Storm first attempt (currently debugging)
@TheGreatfpmK
Added over-approximation option for SAYNT; first analysis of over-app…
8bfdbba 
…roximation values
@TheGreatfpmK
Dynamic threads based on overapproximation
2febef3
@TheGreatfpmK
Finished dynamic threads based on overapproximation
11b74cd
@TheGreatfpmK
working on enhanced SAYNT example for paper
ce1389a
@TheGreatfpmK
New experiments for overapproximation beliefs
1cc8000
TheGreatfpmK and others added 26 commits April 4, 2024 14:55
@TheGreatfpmK
Merge remote-tracking branch 'origin/belief-exploration' into belief-…
507abcb 
…exploration
@TheGreatfpmK
added some sarsop experiments
c74fb7f
@TheGreatfpmK
more discounted experiments
a9c6541
@TheGreatfpmK
Merge remote-tracking branch 'origin/belief-exploration' into belief-…
ba4275c 
…exploration
@TheGreatfpmK
fixed last merge
1511b4c
@TheGreatfpmK
pomdp format export replaces non-existing actions with the first acti…
062bfd1 
…on instead of self-loop
@TheGreatfpmK
fixed pomdp export
ae10103
@TheGreatfpmK
added experimental splitting
cf4daea
@TheGreatfpmK
Merge remote-tracking branch 'origin/belief-exploration' into belief-…
583f12a 
…exploration
@TheGreatfpmK
added option to use trivial cutoff bounds in beliefMCExplorer
dcc7b7f
@TheGreatfpmK
saynt discounted experiments
cd00b3b
@TheGreatfpmK
more discount experiments
65799c6
@TheGreatfpmK
fixed small issue after parse error; added example discounted specifi…
680a1ee 
…cation; added long-traces model
@TheGreatfpmK
added more experiments for table 5
4c6804d
@TheGreatfpmK
added more experiments for table 5
77cc8c2
@TheGreatfpmK
removed all experiments files
83abd78
@TheGreatfpmK
Merge remote-tracking branch 'upstream/master' into belief-exploration
2936bde
@TheGreatfpmK
removed some leftover code
4ec7ae4
@TheGreatfpmK
removed experimental code
8c7a453
@TheGreatfpmK
removed unnecessary line
b8b88b1
@TheGreatfpmK
added user friendly SAYNT CLI option; updated README with new options…
907c3af 
…/examples
@TheGreatfpmK
fixed controller size computation for multiple FSCs
500be2c
@TheGreatfpmK
fixed belief FSC size computation
c003cc8
@TheGreatfpmK
Added some comments
3599b2c
@TheGreatfpmK
Fixed posterior aware export
5016eb5
@TheGreatfpmK
Merge branch 'randriu:master' into belief-exploration
6ff2c9f
@TheGreatfpmK TheGreatfpmK self-assigned this Mar 10, 2025
@TheGreatfpmK TheGreatfpmK added the enhancement New feature or request label Mar 10, 2025
@TheGreatfpmK TheGreatfpmK marked this pull request as draft August 26, 2025 08:46
@TheGreatfpmK TheGreatfpmK mentioned this pull request Oct 5, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@TheGreatfpmK TheGreatfpmK

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@TheGreatfpmK