Drop support for python 3.8

[MichaelFu512]

Pull Request Description

Featuretools and woodwork (and whole lot of other libraries) are dropping support for python 3.8, which will (and is) causing a lot of test run errors.

Closes #4416

---

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.7%. Comparing base (7bd6ed6) to head (3def0d4).
Report is 24 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #4414     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        357     357             
  Lines      39965   39973      +8     
=======================================
+ Hits       39840   39849      +9     
+ Misses       125     124      -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[MichaelFu512]

From #4314, here's the list of ones mentioned that are / have been unpinned and don't seem to cause any issues:

• networkx (unpinned in this MR)
• scikit-learn (unpinned in this MR, had to rework a test in test_partial_dependence.py)
• numpy
• dask
• distributed
• matplotlib
• featuretools
• woodwork

Here's the list of ones that cause issues (test failures):

• pandas
• scipy
• shap

Also, vowpalwabbit according to their documentation (https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Python#support) doesn't support python 3.11 (you'd have to use vowpal-wabbit-next)

[eccabay]

You may need to remove the 3.8 workflows on the github side in order to merge

[eccabay]

Mega nit: drop -> dropped :D

[eccabay]

Also, we should probably consider this a breaking change!

[eccabay]

Why did nlp-primitives disappear?

[MichaelFu512]

If I remember correctly, latest_dependency_checker was not happy having that in there and would fail. I'll try adding it back again and seeing what happens.

[MichaelFu512]

yeah it failed again when I re-added nlp-primitives back into latest-dependency

[thehomebrewnerd]

@MichaelFu512 I think you might want to try to get to the bottom of this. I think the NaturalLanguageFeaturizer depends on nlp-primitives for the PolarityScore and DiversityScore primitives, so it seems like nlp-primitives should be included here.

[thehomebrewnerd]

I'm not up to date on how the dependency checkers are working in EvalML, but the latest version of nlp-primitives is 2.13.0 which was released yesterday and will be installed in your CI runs. Maybe you need to update the version from 2.11.0 to 2.13.0 here instead of removing the line?

[MichaelFu512]

Same error weirdly enough,

Displaying dependencies which have changed, with main on the left and the new branch on the right:
21d20
< nlp-primitives==2.13.0

Normally, if the dependency was out of date, it'd look something like:

Displaying dependencies which have changed, with main on the left and the new branch on the right:
18c18
< matplotlib==3.8.4
---
> matplotlib==3.9.0

I'll have to check why.

[MichaelFu512]

So I think the reason it failed was because of this line in make_deps_diff.sh

pip freeze | grep -v "evalml.git" | grep -E "${allow_list}" > "${DEPENDENCY_FILE_PATH}"

This line compares the requirements gotten from pip freeze | grep -v "evalml.git" to the ones in allow_list (which was made earlier in the file).

allow_list lists nlp-primitives while pip freeze | grep -v "evalml.git" lists nlp_primitives. This difference makes it so that nothing even shows up in the diff since it's looking for nlp_primitives rather than nip-primitives.

I changed the line to:

(pip freeze | grep -v "evalml.git") | tr "_" "-" | grep -E "${allow_list}" > "${DEPENDENCY_FILE_PATH}"

which just converts the "_" to "-".

[jeremyliweishih]

LGTM once Becca's comments are resolved