[cloudflare_logpush.network_analytics] convert_json_ipv4_options fails on multi-value IPv4Options strings (e.g. "7,0")

[andrewkroh]

Integration Name

Cloudflare Logpush [packages/cloudflare_logpush]

Dataset Name

cloudflare_logpush.network_analytics

Integration Version

1.44.0

Agent Version

9.3.4

Agent Output Type

elasticsearch

Elasticsearch Version

Serverless

OS Version and Architecture

Elastic Agent docker image on k8s

Software/API Version

SaaS

Error Message

Processor "convert" with tag convert_json_ipv4_options in pipeline
"logs-cloudflare_logpush.network_analytics-1.44.0" failed with message
"For input string: "7,0""

The processor's on_failure swallows the exception and appends the message
to error.message, so events are not dropped — but cloudflare_logpush.network_analytics.ipv4.options is left unset.

Event Original

{"AttackCampaignID":"","AttackID":"","AttackVector":"","ColoName":"mad01","Datetime":"2026-05-11T11:31:18Z","DestinationASN":13335,"Direction":"ingress","IPDestinationAddress":"203.0.113.243","IPDestinationSubnet":"203.0.113.0/24","IPProtocol":1,"IPSourceAddress":"198.51.100.140","IPSourceSubnet":"198.51.100.0/24","MitigationReason":"","MitigationScope":"","MitigationSystem":"magic-firewall","Outcome":"pass","ProtocolState":"","RuleID":"c301d61435d948009173f78d37ede4e2","RuleName":"","RulesetID":"1e7b7ef15bbc4ca190ebcc24b27a0e98","RulesetOverrideID":"","SampleInterval":100,"SourceASN":1273,"Verdict":"pass","ColoCity":"Madrid","ColoCode":"MAD","ColoCountry":"ES","ColoGeoHash":"ezjqqgdu26nu","DestinationASNName":"Cloudflare, Inc.","DestinationCountry":"US","DestinationGeoHash":"9q8yyf3hhvek","DestinationPort":0,"GREChecksum":0,"GREEtherType":0,"GREHeaderLength":0,"GREKey":0,"GRESequenceNumber":0,"GREVersion":0,"ICMPChecksum":23487,"ICMPCode":0,"ICMPType":8,"IPFragmentOffset":0,"IPHeaderLength":60,"IPMoreFragments":0,"IPProtocolName":"ICMP","IPTTL":57,"IPTTLBuckets":50,"IPTotalLength":124,"IPTotalLengthBuckets":100,"IPv4Checksum":25737,"IPv4DSCP":0,"IPv4DontFragment":1,"IPv4ECN":0,"IPv4Identification":17232,"IPv4Options":"7,0","IPv6DSCP":0,"IPv6ECN":0,"IPv6ExtensionHeaders":"","IPv6FlowLabel":0,"IPv6Identification":0,"SourceASNName":"Vodafone Group PLC","SourceCountry":"PT","SourceGeoHash":"eyckrc9mfpyx","SourcePort":0,"TCPAcknowledgementNumber":0,"TCPChecksum":0,"TCPDataOffset":0,"TCPFlags":0,"TCPFlagsString":"","TCPMSS":0,"TCPOptions":"","TCPSACKBlocks":"","TCPSACKPermitted":0,"TCPSequenceNumber":0,"TCPTimestampECR":0,"TCPTimestampValue":0,"TCPUrgentPointer":0,"TCPWindowScale":0,"TCPWindowSize":0,"UDPChecksum":0,"UDPPayloadLength":0}

What did you do?

Configured the Cloudflare Logpush integration with the network_analytics data stream to pull logs from an S3 bucket (aws-s3 input). Cloudflare emits IPv4 header options as a comma-separated string in theIPv4Options field of network_analytics_logs. When any packet in a sampled flow had more than one IPv4 option, the value sent was e.g."7,0".

What did you see?

The processor convert_json_ipv4_options failed:

integrations/packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml

Lines 616 to 627 in f6689be

	- convert:
	tag: convert_json_ipv4_options
	field: json.IPv4Options
	target_field: cloudflare_logpush.network_analytics.ipv4.options
	if: ctx.json?.IPv4Options != ''
	type: long
	ignore_missing: true
	on_failure:
	- append:
	field: error.message
	value: |-
	Processor "{{{ _ingest.on_failure_processor_type }}}" with tag {{{_ingest.on_failure_processor_tag}}} in pipeline "{{{ _ingest.on_failure_pipeline }}}" failed with message "{{{ _ingest.on_failure_message }}}"

Long.parseLong("7,0") throws NumberFormatException: For input string: "7,0". The on_failure writes to error.message and the target field is not populated. Every event with multiple IPv4 options is affected.

What did you expect to see?

cloudflare_logpush.network_analytics.ipv4.options populated as an array of longs like [7, 0].

Anything else?

Cloudflare doc references.

• Network Analytics Logs reference: https://developers.cloudflare.com/logs/logpush/logpush-job/datasets/account/network_analytics_logs/
• IPv4Options: type string, "List of Options numbers included in the IPv4 packet header."
• IPv6ExtensionHeaders: type string, "List of Extension Header numbers included in the IPv6 packet header."
• TCPOptions: type string, "List of Options numbers included in the TCP packet header."
• TCPSACKBlocks: type string, "List of the SACK Blocks option header in the TCP packet."

All four of these Cloudflare fields are documented as comma-separated string lists.

[infra-vault-gh-plugin-prod]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[github-actions]

tl;dr: The bug is valid and reproducible from code inspection — convert_json_ipv4_options currently treats IPv4Options as a single long, so comma-separated values like "7,0" fail parsing; the fix should mirror the existing TCPSACKBlocks split/normalize/convert pattern.

Recommendation

Update packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml so json.IPv4Options is normalized to a list before numeric conversion (split comma-separated strings, raise scalar to list, then convert to long values). Add a regression pipeline test for "IPv4Options":"7,0" expecting [7,0] at cloudflare_logpush.network_analytics.ipv4.options.

Findings

• Current root cause: convert_json_ipv4_options directly converts json.IPv4Options to long with no pre-split handling.
  • packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml:616-623
• Schema intent indicates list semantics: ipv4.options is documented as “List of Options numbers included in the IPv4 packet header.”
  • packages/cloudflare_logpush/data_stream/network_analytics/fields/fields.yml:202-204
• Existing in-pipeline precedent: TCPSACKBlocks already uses robust list handling (split + script normalization + foreach append + convert).
  • packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml:850-889
• Test coverage gap: pipeline fixture includes IPv4Options as scalar (1) or empty string, but no CSV case like "7,0".
  • Input fixture: packages/cloudflare_logpush/data_stream/network_analytics/_dev/test/pipeline/test-pipeline-network-analytics.log:1-5
  • Expected fixture shows scalar output for existing cases: .../test-pipeline-network-analytics.log-expected.json:84-90
• Related prior fixes (same bug class family):
  • Issue #14096 fixed by PR #14099 (field rename/empty handling)
  • Issue #14233 fixed by PR #14234 (TCP SACK CSV list handling)
  • The current IPv4Options path remained scalar-convert and was not covered by those fixes.

Verification

I ran these checks locally:

$ nl -ba packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml | sed -n '612,627p;846,889p'
...
616  - convert:
617      tag: convert_json_ipv4_options
618      field: json.IPv4Options
619      target_field: cloudflare_logpush.network_analytics.ipv4.options
620      if: ctx.json?.IPv4Options != ''
621      type: long
...
850  - split:
851      field: json.TCPSACKBlocks
...
875  - foreach:
...
884  - convert:
885      tag: convert_tcp_sack_blocks
886      field: cloudflare_logpush.network_analytics.tcp.sack.blocks
887      if: ...
888      type: long

$ nl -ba packages/cloudflare_logpush/data_stream/network_analytics/fields/fields.yml | sed -n '198,206p;320,331p'
...
202  - name: options
203    type: long
204    description: List of Options numbers included in the IPv4 packet header.
...
328  - name: blocks
329    type: long

$ elastic-package test pipeline -v -d packages/cloudflare_logpush/data_stream/network_analytics
bash: elastic-package: command not found

And from git history for related fixes:

$ git --no-pager log --oneline -- packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml | head -n 5
2a2f24bd3f cloudflare_logpush: fix several ingest pipeline and field definition bugs across multiple data streams (#18673)
...
4a0d07761f cloudflare_logpush: fix handling of TCP SACK block lists (#14234)
ad74f89d3d cloudflare_logpush: fix handling Cloudflare's field renames (#14099)

Detailed Action Plan

1. In packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml, replace the current scalar-only convert_json_ipv4_options block (616-623) with a list-safe flow modeled on TCPSACKBlocks (850-889):
   1. Split json.IPv4Options on , when it is a non-empty string.
   2. Script-normalize non-list non-empty values into a one-element array.
   3. Append each element into cloudflare_logpush.network_analytics.ipv4.options (deduplicated like existing list processors).
   4. Convert cloudflare_logpush.network_analytics.ipv4.options to long.
2. Add regression input to packages/cloudflare_logpush/data_stream/network_analytics/_dev/test/pipeline/test-pipeline-network-analytics.log with "IPv4Options":"7,0".
3. Update expected output in .../test-pipeline-network-analytics.log-expected.json so cloudflare_logpush.network_analytics.ipv4.options is [7,0].
4. Run package pipeline tests in an environment with elastic-package available.
5. Add changelog entry in packages/cloudflare_logpush/changelog.yml referencing this fix.

Related Items

Type	Link	Relevance
Issue	#18973	Current bug report (IPv4Options CSV parse failure)
Issue	#14096	Earlier network_analytics parse/rename handling bug
PR	#14099	Fix for #14096; did not add IPv4Options CSV handling
Issue	#14233	Earlier CSV parse failure for TCPSACKBlocks
PR	#14234	Implemented split/normalize/convert pattern for TCP SACK lists
PR	#18673	Later cleanup in same pipeline; no IPv4Options list parsing fix
File	packages/cloudflare_logpush/data_stream/network_analytics/elasticsearch/ingest_pipeline/default.yml	Location of failing and reference processors
File	packages/cloudflare_logpush/data_stream/network_analytics/fields/fields.yml	Field type/description for ipv4.options
File	packages/cloudflare_logpush/data_stream/network_analytics/_dev/test/pipeline/test-pipeline-network-analytics.log	Pipeline input fixtures missing "7,0" case
File	packages/cloudflare_logpush/data_stream/network_analytics/_dev/test/pipeline/test-pipeline-network-analytics.log-expected.json	Expected output fixtures to extend

---

What is this? | From workflow: Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.