Fixup notebook tests#933
Merged
Merged
Conversation
Collaborator
eric-anderson
commented
Oct 16, 2024
eric-anderson
commented
- Add script to run tests that will
- optionally run in both local and ray mode
- split between fast and slow tests
- verify all notebooks are accounted
- Fix file scans so they can run with invalid AWS credentials in environment, same trick as for materialize, try being anonymous if default doesn't work.
- Fix ndd_example notebook to not specify parallelism.
- Fix tutorial notebook to use materialize to avoid some re-execution
* Add script to run tests that will * optionally run in both local and ray mode * split between fast and slow tests * verify all notebooks are accounted * Fix file scans so they can run with invalid AWS credentials in environment, same trick as for materialize, try being anonymous if default doesn't work. * Fix ndd_example notebook to not specify parallelism. * Fix tutorial notebook to use materialize to avoid some re-execution
bsowell
approved these changes
Oct 16, 2024
Contributor
bsowell
left a comment
bsowell
left a comment
There was a problem hiding this comment.
Fine with this as long as you coordinate with Bohou on the file scan stuff.
| else: | ||
| return all(path.startswith("s3:") for path in self._paths) | ||
|
|
||
| def _try_infer_fs(self): |
Contributor
There was a problem hiding this comment.
This seems related/overlapping with #925. Are they going to conflict?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.