Finding the metadata JSON to fully populate the stations JSON and GeoJSON is not possible:
- When the method is called and the S3 store is active it will only look for
metadata.json on S3 (not locally) when populating stations.json and stations.geojson. This is due to how S3.read is written
- The
_get_old_metadata child function was hard coded to pass stations.json as the file basename to self.metadata_handler.latest_metadata, meaning the desired metadata.json can’t be found
- When using
latest_metadata on a Local store it doesn’t appear to populate the last_local_output_directory kwarg to Local.latest_metadata. This results in self.metadata_by_filesystem being passed only the “metadata.json” string without a directory, triggering a TypeError: expected str, bytes or os.PathLike object, not NoneType
As a final minor note, the name is a bit confusing since it also creates a JSON
Note that this is a blocking issue for a downstream ETL
Finding the metadata JSON to fully populate the stations JSON and GeoJSON is not possible:
metadata.jsonon S3 (not locally) when populatingstations.jsonandstations.geojson. This is due to howS3.readis written_get_old_metadatachild function was hard coded to passstations.jsonas the file basename toself.metadata_handler.latest_metadata, meaning the desiredmetadata.jsoncan’t be foundlatest_metadataon a Local store it doesn’t appear to populate thelast_local_output_directorykwarg toLocal.latest_metadata. This results inself.metadata_by_filesystembeing passed only the “metadata.json” string without a directory, triggering aTypeError: expected str, bytes or os.PathLike object, not NoneTypeAs a final minor note, the name is a bit confusing since it also creates a JSON
Note that this is a blocking issue for a downstream ETL