Similar functionality to pa.checks.Check.isin but opposite direction?

[johnkangw]

Hi,
I looked around on the discussions and documentation and could not find if there was pre-built in functionality to do something like the pa.checks.Check.isin but in the opposite direction. What I mean by that is can I check that all of the values from the allowed_values is present in the column? I think the current check makes sure all of the values in the column are only allowed to be values from the allowed_values parameter.

e.g.: I have a df of different names
df = pd.DataFrame({"a": [fred, joe, john, bob]})
I also have a separate list of names that must be present
list_names_must_be_present = [fred, joe]
I would like a pa.checks that checks to ensure all of the values from the list are present in the column I am checking. Does that functionality exist? If not, is there a way to check this? I've used pytest in the past to check this, but not sure if I can do this with Pandera.

[cosmicBboy]

did you see https://pandera.readthedocs.io/en/stable/reference/generated/methods/pandera.checks.Check.notin.html#pandera.checks.Check.notin

[cosmicBboy]

oh, I see what you mean by opposite direction. No, currently this check is not supported... wanna make a PR for it? 🙂

[cosmicBboy]

the custom check would be fairly simple:

values = pd.Series(["fred", "joe"])
check = pa.Check(lambda series: values.isin(series).all())

If you're up for it, adding a built-in method to pa.Check would be simply adapting the above function here.

[johnkangw]

@cosmicBboy I had a few problems with getting this to work but it was data error (whitespace!).

The key change I made to your code was adding in the .str.strip() method. Should we add a parameter to perform this automatically?

check_sites = pa.Check( lambda series: sites_to_ensure.isin(series.str.strip()).all())

[johnkangw]

@cosmicBboy Sure! Do you mind if I take a crack at a PR this evening / tomorrow morning? I probably will need some of your help with the PR as I won't get it right the first time!