Implemented algorithms¶
Model |
base learner |
meta learner |
Schedule |
|||||
algorithm |
opt. |
surr. |
algorithm |
opt. |
surr. |
|||
static |
OGD [Zin03] |
OGD |
— |
— |
— |
— |
— |
— |
BGD1 [FKM05] |
BGD1 |
— |
— |
— |
— |
— |
— |
|
BGD2 [ADX10] |
BGD2 |
— |
— |
— |
— |
— |
— |
|
OEGD [CYL+12] |
OEGD |
✔ |
— |
— |
— |
— |
— |
|
OptimisticOGD [RS13] |
OptimisticOGD |
✔ |
— |
— |
— |
— |
— |
|
SOGD [OPal18] |
SOGD |
— |
— |
— |
— |
— |
— |
|
dynamic |
Ader [ZLZ18] |
OGD |
— |
— |
Hedge |
— |
— |
SSP |
Ader++ [ZLZ18] |
OGD |
— |
✔ |
Hedge |
— |
✔ |
SSP |
|
PBGD1 [ZWZZ21] |
OGD |
— |
✔ |
Hedge |
— |
✔ |
SSP |
|
PBGD2 [ZWZZ21] |
OGD |
— |
✔ |
Hedge |
— |
✔ |
SSP |
|
Sword [ZZZZ20] |
OEGD + OGD |
✔ |
— |
OptimisticHedge |
✔ |
✔ |
SSP |
|
Sword++ [ZZZZ21] |
OptimisticOGD |
✔ |
✔ |
OptimisticHedge |
✔ |
✔ |
SSP |
|
Scream [ZWZ22] |
OGD |
— |
✔ |
Hedge |
— |
✔ |
SSP |
|
adaptive |
AFLH [HS07] |
BlackBox |
— |
— |
AFLH Meta |
— |
— |
CGC |
AFLH++ [WZZ18] |
BlackBox |
— |
✔ |
AFLH Meta |
— |
✔ |
CGC |
|
SAOL [DGSS15] |
BlackBox |
— |
— |
Prod |
— |
— |
GC |
|
SACS [ZLZ19] |
SOGD |
— |
— |
AdaNormalHedge |
— |
— |
CGC |
|
PSACS [ZLZ19] |
SOGD |
— |
— |
AdaNormalHedge |
— |
— |
PCGC |
|
Note
One can combine base, meta, schedule, specification freely to
define more algorithms!