Category: Uncategorized

I have not seen this cool leaderboard for AI challenges before.
https://leaderboard.allenai.org/

There are a few very interesting similar competition leaderboards for machine learning such as Kaggle and Numerai. Allenai host right now 4 interesting NLP challenges.

Here is the description of one of the challenges:

OpenBookQA: Open Book Question Answering

OpenBookQA is a new kind of question-answering dataset modeled after open book exams for assessing human understanding of a subject. It consists of 5,957 multiple-choice elementary-level science questions (4,957 train, 500 dev, 500 test), which probe the understanding of a small “book” of 1,326 core science facts and the application of these facts to novel situations. For training, the dataset includes a mapping from each question to the core science fact it was designed to probe. Answering OpenBookQA questions requires additional broad common knowledge, not contained in the book. The questions, by design, are answered incorrectly by both a retrieval-based algorithm and a word co-occurrence algorithm. Strong neural baselines achieve around 50% on OpenBookQA, leaving a large gap to the 92% accuracy of crowd-workers.

Jeremy Howard et al, at fast.ai has done what one might consider a huge breakthrough in regards to training deep learning models quickly.

They managed to train Imagenet in 18 minutes using publicly available resources that only cost them $40 to run!

this was their method:

• fast.ai’s progressive resizing for classification, and rectangular image validation
• NVIDIA’s NCCL with PyTorch’s all-reduce
• Tencent’s weight decay tuning; a variant of Google Brain’s dynamic batch sizes, gradual learning rate warm-up (Goyal et al 2018, and Leslie Smith 2018).
• ResNet-50 architecture
• SGD with momentum.

http://www.fast.ai/2018/08/10/fastai-diu-imagenet/