We need to build some of the index-provider and indexer internal as standalone tools. Right now, it is very difficult to predict what went wrong if error is not very clear. Case in point, we get an error during ingestion saying there is a hash mismatch. Now, I have to open the DB to try to figure out what chunks were generate when this ad was originally created. We should build tools which can provide us some insight into how output will be impacted based on different inputs to try to predict possible error scenarios.
Example: Chunker, MHlister based on file etc
I can contribute to this issue as time permits.
We need to build some of the index-provider and indexer internal as standalone tools. Right now, it is very difficult to predict what went wrong if error is not very clear. Case in point, we get an error during ingestion saying there is a
hash mismatch. Now, I have to open the DB to try to figure out what chunks were generate when this ad was originally created. We should build tools which can provide us some insight into how output will be impacted based on different inputs to try to predict possible error scenarios.Example: Chunker, MHlister based on file etc
I can contribute to this issue as time permits.