We present fast, accurate, direct non-projective dependency parsers with third-order features. Our approach uses AD3, an accelerated dual decomposition algorithm which we extend to handle specialized head automata and sequential head bigram models. Experiments in fourteen languages yield parsing speeds competitive to projective parsers, with state-of-The-art accuracies for the largest datasets (English, Czech, and German).
