Feature Proposal: Contextual de/serialization

Yin wrote:
"the request is about contextual de/serialization.

specifically, there are fields that we want to exclude from
de/serializing in some contexts. one example is the downloadStatus
field, which is useful only on the server. another example is
localLocation, say for cached images, which can have different values
on different users' machines.

the proposal is to have 2 new simpl annotations for a field to specify
applicable contexts, in the form of context names in strings. when
de/serializing, you specify what's the context in the
TranslationContext object, and simpl will determine if the field
should be de/serialized according to the context annotations.

new annotations:

• @simpl_context(String[] contextNames): only de/serialize this field
  when one of the contextNames is used for the TranslationContext
  object.
• @simpl_no_context(String[] contextNames): do not de/serialize this
  field when on of the contextNames is used for the TranslationContext
  object.

you can see that they are kinda complementary.

an example:

// in class Document:
@simpl_context("cache")  // only de/serialize in the "cache" context
private DownloadStatus downloadStatus;

// at the time of de/serialization outside of caches:
// doc1.downloadStatus is serialized and sent to the client:
Document doc1 = getDoc();
translationContext.addContext("cache"); // here!
msg = serialize(doc1, Format.JSON, translationContext);
sendToClient(msg);

// doc2.downloadStatus does NOT get deserialized in the client:
String msg = recvFromServer();
Document doc2 = deserialize(serializedDocument, Format.JSON,
defaultTranslationContext);

the implementation seems straightforward to me. what do you think?"

A few questions to start:

1. You had the example of adding a context ... translationContext.addContext("cache");
   Does this imply that a translationContext can have multiple contexts?

2. If yes: How will multiple contexts inter-operate? What happens if the contexts for a field can be excluded in a combination of the contexts?

3. I get the idea behind this, and I like the idea behind it, but the API scares me. It seems like these contexts can be easy to miss or forget about. Are there any better ways to express this idea? How will you prevent data loss?

3.1) Should "context" thus become a first-level parameter for serialization and deserialization? That is you won't be able to de/serialize without first specifying that you're in either the default context or some other context?

Maybe: simpl.serialize(object).context("contextname")...?

3.2) How will this interoperate w/ Descriptors? Will the complete description for an object contain every field for every context, or will it contain only those things relevant for the given context?

4. Can you exclude a class in a context?

5. I don't like the name of the annotation. Is there a better way to reword it to make it more clear that this excludes / includes a field in a givne "context" name?

6. Do translated representations in source code contain all fields in all contexts, or will translation be retooled to fit the context model? (That is a class A with fields of context {a,b,c} will translate differently for different contexts a, b, c.)

7. Are there other ways we can use this to simplify the api's ? Maybe a context for mmd?

8. How will this work across implementations? Will implementers have to respect all context annotations, or can they treat them as they please?

9. What is the purpose of "context"? Can it be achieved other ways?

@quyin @andruidk @rhema etc... Helloooo?

[quyin]

1. yes, I think there could be orthogonal contexts, such as "service" + "cache". in some sense they are more like tags.

2. it is possible to do inclusion / exclusion based on multiple contexts, but I don't have a specific design yet.

3. the simpl_context tag was designed so that that if you do not use it at all, the field always gets de/serialized to prevent data loss. the simpl_no_context may be problematic; the mechanism for negation might need to be thought of more.

3.1) the fluent API looks great to me. I like the idea.

3.2) yes the context should include all the context tags.

4. not right now.

5. simpl_cond has been proposed. yes we can brainstorm on the annotation name. simpl_in_context?

6. no. for now I'm only thinking of conditional translation of values, not the schema itself.

7. I don't understand the question very well.

8. with its current spec I think it is easy to do conditional de/serialization, once you can provide some meta-information to the implementation. so I would say they should respect the context annotations.

9. the purpose is for conditional de/serializations, with use cases I explained in the first email. there can certainly be other ways, and this was proposed because it looks straight-forward. any new idea is definitely welcomed!

[andru1d]

You had the example of adding a context ...
translationContext.addContext("cache");
Does this imply that a translationContext can have multiple contexts?

it does imply that. we don't have to enable that, but i don't think its
much more work or will effect performance much.

Should "context" thus become a first-level parameter for serialization
and deserialization? That is you won't be able to de/serialize without
first specifying that you're in either the default context or some other
context?
No. this is a fairly esoteric feature. people don't usually need it.

i actually suggested @simpl_cond & @simpl_no_cond
for conditional.

the point is that sometimes a field should be de/serialized under some
circumstances, and not in others. we have several use cases. they are
important to work being performed in the lab.

the semantics yin & i have defined for this are straightforward to
implement and easy to use for developers.

the reason to use methods on the de/serialization context, is that this
mechanism already exists. it already does a bunch of important things. the
mechanism enables various functionalities that affect how de/serialization
works at run time, rather than compile time.

relative URLs are an example of another functionality that is implemented
with SerializationContext.

ok, @tom?

andruid

andruid

On Thu, May 2, 2013 at 12:43 AM, Tom White notifications@github.com wrote:

A few questions to start:

You had the example of adding a context ...
translationContext.addContext("cache");
Does this imply that a translationContext can have multiple contexts?
2.

If yes: How will multiple contexts inter-operate? What happens if the
contexts for a field can be excluded in a combination of the contexts?
3.

I get the idea behind this, and I like the idea behind it, but the API
scares me. It seems like these contexts can be easy to miss or forget
about. Are there any better ways to express this idea? How will you prevent
data loss?

3.1: Should "context" thus become a first-level parameter for
serialization and deserialization? That is you won't be able to
de/serialize without first specifying that you're in either the default
context or some other context?

Maybe: simpl.serialize(object).context("contextname")...?

3.2: How will this interoperate w/ Descriptors? Will the complete
description for an object contain every field for every context, or will it
contain only those things relevant for the given context?

Can you exclude a class in a context?
2.

I don't like the name of the annotation. Is there a better way to
reword it to make it more clear that this excludes / includes a field in a
givne "context" name?
3.

Do translated representations in source code contain all fields in all
contexts, or will translation be retooled to fit the context model? (That
is a class A with fields of context {a,b,c} will translate differently for
different contexts a, b, c.)
4.

Are there other ways we can use this to simplify the api's ? Maybe a
context for mmd?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/6#issuecomment-17313168
.

andruid kerne, ph.d.
director, interface ecology lab
associate professor, department of computer science and engineering
texas a&m university 979.862.3684 fax
college station, tx 77843-3112 http://ecologylab.net

http://facebook.com/ecologylab

Interfaces are the multidimensional border zones through which the
interdependent relationships of people, activities, codes, components,
and systems are constituted. Interface ecology investigates the
dynamic interactions of media, cultures, and disciplines that
flow through interfaces.

So, this whole concept would be pretty straightforward to implement w/ the interpretation architecture... just introduce a filter at each end at the simpl level that filters out interpretations that don't belong in your context. Cool.

My concern is that if the whole purpose of the context is to add additional information to an object, it would make more sense to use other idiomatic methods to get the same value. Contexts add a lot more conceptual overhead than simply using OOP.

In the document example:

I have a document:

class Document {
 // Fields for a document
}

If I want to represent a document that has been Downloaded at a certain time, or has been cached, I could

1. Add those fields to the document

class Document { 
// Fields for a document
// Fields for a cached document 
// Fields for a local document 
}

or
2. Add new document classes that inherit from Document and add their own information .

@simpl_inherit
class CachedDocument extends Document{
// Cache info
}
@simpl_inherit
class LocalDocument extends Document 
{
}

If the class had a nice polymorphic interface, chances are consumers of different documents wouldn't need to know about the specifics of caching and local location, etc. That gets handled in the subclasses of document w/ specific information . (Polymoprhism wins, that is)

"Contextual" information then becomes "what type of Document am I expecting here" -> Which is a polymorphism problem, and not a subsetting/context problem.

Polymorphism seems like a nicer way to address the problem at hand (Adding additional information for different semantic types of data) It's more idiomatic and matches our general commitment to OO principles. This seems to follow from the fact that in discussing the key use cases, we're discussing semantically different kinds of documents.

This is why I asked question 6: In my mind, if you're thinking that you would ever want to represent different objects in different contexts, probably you just meant to use different objects in the first place.

The type of objects we end up making will also be more usable if we prefer polymorphism over shoe-horning additional data fields into classes that may not carry the appropriate semantic meanings.

If you feel that context is still the right way to solve this problem, let's implement it.

At the moment, I'm not convinced.

[andru1d]

We are sure. Again, the point is that some fields need different treatment
based on the context of de/serialization. different fields would be
completely wrong, if it's not clear to you, @tom, please spend some time
learning about the use cases. Or just trust us.

On Friday, May 3, 2013, Tom White wrote:

So, this whole concept would be pretty straightforward to implement w/ the
interpretation architecture... just introduce a filter at each end at the
simpl level that filters out interpretations that don't belong in your
context. Cool.

My concern is that if the whole purpose of the context is to add
additional information to an object, it would make more sense to use other
idiomatic methods to get the same value. Contexts add a lot more conceptual
overhead than simply using OOP.

In the document example:

I have a document:

class Document {
// Fields for a document}

If I want to represent a document that has been Downloaded at a certain
time, or has been cached, I could

1. Add those fields to the document

~~java
class Document {
// Fields for a document
// Fields for a cached document
// Fields for a local document
}

or
2. Add new document classes that inherit from Document and add their own information .

@simpl_inherit
class CachedDocument extends Document{
// Cache info
}
@simpl_inherit
class LocalDocument extends Document
{
}

If the class had a nice polymorphic interface, chances are consumers of
different documents wouldn't need to know about the specifics of caching
and local location, etc. That gets handled in the subclasses of document w/
specific information . (Polymoprhism wins, that is)

"Contextual" information then becomes "what type of Document am I
expecting here" -> Which is a polymorphism problem, and not a
subsetting/context problem.

Polymorphism seems like a nicer way to address the problem at hand (Adding
additional information for *different semantic types of data*) It's more
idiomatic and matches our general commitment to OO principles. This seems
to follow from the fact that in discussing the key use cases, we're
discussing semantically different kinds of documents.

This is why I asked question 6: In my mind, if you're thinking that you
would ever want to represent different objects in different contexts,
probably you just meant to use different objects in the first place.

The type of objects we end up making will also be more usable if we prefer
polymorphism over shoe-horning additional data fields into classes that may
not carry the appropriate semantic meanings.

If you feel that context is still the right way to solve this problem,
let's implement it.

At the moment, I'm not convinced.

—
Reply to this email directly or view it on GitHub<https://github.com/ecologylab/simpl/issues/6#issuecomment-17422826>
.

andruid kerne, ph.d.
director, interface ecology lab
associate professor, department of computer science and engineering
texas a&m university 979.862.3684 fax
college station, tx 77843-3112 http://ecologylab.net

http://facebook.com/ecologylab

Interfaces are the multidimensional border zones through which the
interdependent relationships of people, activities, codes, components,
and systems are constituted. Interface ecology investigates the
dynamic interactions of media, cultures, and disciplines that
flow through interfaces.