How to integrate Adaptive Gradmatch in the existing codebase?

[animesh-007]

Hi. Thank you for the great work. I wanted to ask, is there a way where I can use adaptive gradmatch in the codebase? If not, can you guide me on how to integrate it into the existing codebase?

[Chengcheng-Guo]

Hi! Unfortunately, it does not select adaptive coresets currently. DeepCore is developed on the setting that a fixed coreset $\mathcal{S}$ is extracted from $\mathcal{T}$, and we train our model $\theta_\mathcal{S}$ with this specific coreset $\mathcal{S}$, instead of a series of adaptive coresets.

Here is the way to introduce adaptive coresets to DeepCore. First, in class CoresetMethod (deepcore/methods/coresetmethod.py), it was designed that the select method would be only called once and then would return the non-adaptive coreset. For compatibility with adaptive coresets, there is a need to design a new class, e.g., AdaptiveCoresetMethod, where the select method is designed to be called iteratively and keep returning adaptive coresets as the training progresses. Second, in main.py, select was called only once before the start of training. To integrate adaptivity, select should be called every epoch.

Above are my ideas on how to include adaptivity into it. Hopefully they will be very helpful!