hypothesis

active

hypothesis:models-trained-directly-with-asynchronous-updates-would-exhibit-even-greater-robustness-than-synchronously-trained-models

Models trained directly with asynchronous updates would exhibit even greater robustness than synchronously trained models

Hypothesis that motivated the asynchronous robustness comparison experiment

Source paper

extracted_from

Differentiable Logic Cellular Automata: From Game of Life to pattern generation with learned recurrent circuits

Neighborhood — ranked by edge-count

Findings (1)

finding

Asynchronously trained DiffLogic CA shows greater robustness to 10x10 pixel damage than synchronously trained version, measured by sum of absolute differences
associated_withsupports
Quantitative comparison of synchronous vs asynchronous training for noise resilience

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Asynchronous training improves robustness of DiffLogic CA circuits to perturbations compared to synchronous trainingclaim0.839
Interpretive claim supported by damage resilience experiment results
Asynchronous Update Trainingmethod0.788
Training regime where random subsets of cells update per step, improving robustness of learned circuits
We hypothesize that native self-report, fine-tuned introspection models, and trained activation-to-language systems will show different performance on bias-resistant localization and strength benchmarkshypothesis0.786
Comparative prediction motivating future work contrasting different approaches to LLM self-knowledge
RL teaches the model to comply even when unmonitored on the training prompt through non-robust heuristics that do not generalizehypothesis0.781
Hypothesis explaining why the compliance gap decreases but is recovered by small prompt modifications
There are fewer representations competent for N tasks than M<N tasks, so training more general models should yield fewer possible solutionshypothesis0.776
Selective pressure toward convergence via task generality
A vision model trained on ImageNet can be aligned with a model trained on Places-365 while maintaining good performance, and early layers are more interchangeable than later layersfinding0.771
Lenc & Vedaldi result illustrating data independence in representations and layer-wise alignment
Jointly training a language model with a vision model improves performance on language tasks compared to training the language model alonefinding0.767
OpenAI GPT-4V finding supporting cross-modal training benefit
Synchronously trained DiffLogic CA circuit succeeds at asynchronous inference without retrainingfinding0.765
Unexpected result demonstrating robustness of learned circuits beyond their training regime