Skip to content

Allow TokenFilter to preserve empty #716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nik9000 wants to merge 1 commit into from
Closed

Allow TokenFilter to preserve empty #716

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 60 additions & 4 deletions src/main/java/com/fasterxml/jackson/core/filter/FilteringParserDelegate.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,16 +273,22 @@ public JsonToken nextToken() throws IOException
_exposedContext = null;
if (ctxt.inArray()) {
t = delegate.getCurrentToken();
// Is this guaranteed to work without further checks?
// if (t != JsonToken.START_ARRAY) {
_currToken = t;
if (_currToken == JsonToken.END_ARRAY) {
_headContext = _headContext.getParent();
_itemFilter = _headContext.getFilter();
}
return t;
}

// 19-Jul-2021, tatu: [core#700]: following was commented out?!
// Almost! Most likely still have the current token;
// with the sole exception of FIELD_NAME
t = delegate.currentToken();
if (t == JsonToken.END_OBJECT) {
_headContext = _headContext.getParent();
_itemFilter = _headContext.getFilter();
}
if (t != JsonToken.FIELD_NAME) {
_currToken = t;
return t;
Expand Down Expand Up @@ -562,12 +568,15 @@ protected final JsonToken _nextToken2() throws IOException
continue main_loop;

case ID_END_ARRAY:
case ID_END_OBJECT:
{
boolean returnEnd = _headContext.isStartHandled();
f = _headContext.getFilter();
if ((f != null) && (f != TokenFilter.INCLUDE_ALL)) {
boolean includeEmpty = f.includeEmptyArray(_headContext.hasCurrentIndex());
f.filterFinishArray();
Copy link
Copy Markdown
Contributor Author

@nik9000 nik9000 Sep 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in the old code this was calling the wrong callback here.

if (includeEmpty) {
return _nextBuffered(_headContext);
}
}
_headContext = _headContext.getParent();
_itemFilter = _headContext.getFilter();
Expand All @@ -576,6 +585,23 @@ protected final JsonToken _nextToken2() throws IOException
}
}
continue main_loop;
case ID_END_OBJECT:
{
boolean returnEnd = _headContext.isStartHandled();
f = _headContext.getFilter();
if ((f != null) && (f != TokenFilter.INCLUDE_ALL)) {
boolean includeEmpty = f.includeEmptyArray(_headContext.hasCurrentName());
f.filterFinishObject();
if (includeEmpty) {
return _nextBuffered(_headContext);
} }
_headContext = _headContext.getParent();
_itemFilter = _headContext.getFilter();
if (returnEnd) {
return (_currToken = t);
}
}
continue main_loop;

case ID_FIELD_NAME:
{
Expand Down Expand Up @@ -708,13 +734,16 @@ protected final JsonToken _nextTokenWithBuffering(final TokenFilterContext buffR
continue main_loop;

case ID_END_ARRAY:
case ID_END_OBJECT:
{
// Unlike with other loops, here we know that content was NOT
// included (won't get this far otherwise)
f = _headContext.getFilter();
if ((f != null) && (f != TokenFilter.INCLUDE_ALL)) {
boolean includeEmpty = f.includeEmptyArray(_headContext.hasCurrentIndex());
f.filterFinishArray();
if (includeEmpty) {
return _nextBuffered(buffRoot);
}
}
boolean gotEnd = (_headContext == buffRoot);
boolean returnEnd = gotEnd && _headContext.isStartHandled();
Expand All @@ -727,6 +756,33 @@ protected final JsonToken _nextTokenWithBuffering(final TokenFilterContext buffR
}
}
continue main_loop;
case ID_END_OBJECT:
{
// Unlike with other loops, here we know that content was NOT
// included (won't get this far otherwise)
f = _headContext.getFilter();
if ((f != null) && (f != TokenFilter.INCLUDE_ALL)) {
boolean includeEmpty = f.includeEmptyObject(_headContext.hasCurrentName());
f.filterFinishObject();
if (includeEmpty) {
_headContext._currentName = _headContext._parent == null
? null
: _headContext._parent._currentName;
_headContext._needToHandleName = false;
return _nextBuffered(buffRoot);
}
}
boolean gotEnd = (_headContext == buffRoot);
boolean returnEnd = gotEnd && _headContext.isStartHandled();

_headContext = _headContext.getParent();
_itemFilter = _headContext.getFilter();

if (returnEnd) {
return t;
}
}
continue main_loop;

case ID_FIELD_NAME:
{
Expand Down
8 changes: 8 additions & 0 deletions src/main/java/com/fasterxml/jackson/core/filter/TokenFilter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,14 @@ public boolean includeEmbeddedValue(Object value) {
return _includeScalar();
}

public boolean includeEmptyArray(boolean contentsFiltered) {
return false;
}

public boolean includeEmptyObject(boolean contentsFiltered) {
return false;
}

/*
/**********************************************************
/* Overrides
Expand Down
20 changes: 20 additions & 0 deletions src/main/java/com/fasterxml/jackson/core/filter/TokenFilterContext.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,16 @@ public TokenFilterContext closeArray(JsonGenerator gen) throws IOException
{
if (_startHandled) {
gen.writeEndArray();
} else {
if ((_filter != null) && (_filter != TokenFilter.INCLUDE_ALL)) {
if (_filter.includeEmptyArray(hasCurrentIndex())) {
if (_parent != null) {
_parent._writePath(gen);
}
gen.writeStartArray();
gen.writeEndArray();
}
}
}
if ((_filter != null) && (_filter != TokenFilter.INCLUDE_ALL)) {
_filter.filterFinishArray();
Expand All @@ -244,6 +254,16 @@ public TokenFilterContext closeObject(JsonGenerator gen) throws IOException
{
if (_startHandled) {
gen.writeEndObject();
} else {
if ((_filter != null) && (_filter != TokenFilter.INCLUDE_ALL)) {
if (_filter.includeEmptyObject(hasCurrentName())) {
if (_parent != null) {
_parent._writePath(gen);
}
gen.writeStartObject();
gen.writeEndObject();
}
}
}
if ((_filter != null) && (_filter != TokenFilter.INCLUDE_ALL)) {
_filter.filterFinishObject();
Expand Down
6 changes: 5 additions & 1 deletion src/test/java/com/fasterxml/jackson/core/BaseTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,11 +451,15 @@ protected String readAndWrite(JsonFactory f, JsonParser p) throws IOException
g.disable(JsonGenerator.Feature.AUTO_CLOSE_JSON_CONTENT);
try {
while (p.nextToken() != null) {
System.err.println(p.currentToken() + " " + p.currentName() + " " + p.currentValue());
Copy link
Copy Markdown
Contributor Author

@nik9000 nik9000 Sep 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left this in to remind myself to ask about currentName on the tokens to close objects. Have a look at what this prints in some of the tests. I'm not sure that currentName has the value you'd expect there. I don't think I use it in that context, but I wanted to bring it up.

Copy link
Copy Markdown
Member

@cowtowncoder cowtowncoder Oct 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just make sure this gets removed before merging -- no System.err or System.out output from Jackson code ever unless absolutely required.

g.copyCurrentEvent(p);
}
} catch (IOException e) {
g.flush();
fail("Unexpected problem during `readAndWrite`. Output so far: '"+sw+"'; problem: "+e);
throw new AssertionError(
"Unexpected problem during `readAndWrite`. Output so far: '" +
sw + "'; problem: " + e.getMessage(),
e);
Copy link
Copy Markdown
Contributor Author

@nik9000 nik9000 Sep 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't required at all but it sure made all the failures easier to see. I had a lot of failures.....

Copy link
Copy Markdown
Member

@cowtowncoder cowtowncoder Oct 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But isn't there already fail() call there? Is this to get stack trace or ... ?

}
p.close();
g.close();
Expand Down
Loading