I am creating a subclass of shelve allowing for searching for partial keys (e.g., key begins with substring, or key ends with substring), and the way that seems most intuitive is to use the db.keys() method, to get a list of keys, and the iterate through them testing for the partial key.
I am doing this to have a searchable multi-language dictionary within a program which returns multiple matching/related words for each word (I currently implement this by returning a list of values for a single key).
Because I intend to use this subclass and its database to hold large amounts of keys as a language dictionary, I believe this approach will be slow (relatively that is. I would like for no lag to be experienced) when iterating keys in numbers possibly exceeding hundreds of thousands.
To account for this slowness and waste of computation in iterating over the entire dictionary, I would want to have an easy way to only look at keys which have the possibility of having a matching key.
I know python has
filter, but it would seem like when calling this filter it would have to iterate over the list to find which items match the filter, and I might as well do my iteration if that is the case.
Does anyone have any ideas?
I am thinking I could return sorted(db.keys) and make a "key" to hold the ranges of each word beginning with a respective letter (i.e., the letter "a" begins lines 0 through 56), and iterate over that range, so I could focus in that way, but this would only be useful when searching for keys beginning with a certain substring, and not for searching for keys ending with one or containing one.
I am using shelve because it can store lists as a value, but I am open to using any other type of data structure which supports this, if anyone knows of an alternate method to do this which would be more effective.