A data driven machine learning based approach is presented to model the constitutive response of anisotropic elasto-damage materials such as wood and fiber reinforced composites. Adopting principles of continuum damage mechanics, the nonlinear constitutive response is formulated by recourse to damage variables or damage tensors. The knowledge of the damage tensor and the undamaged (initial) linear elasticity determines the nonlinear relation between the stress and stain tensors. Instead of analytically expressing the relation between damage and strain state, an artificial neural network (ANN) is employed. For training, testing, and validation of various ANN architectures data sets are generated by Finite Element Method simulations. Monotonous increasing, radial strain loads are applied to sample the strain space with sufficient resolution. Complementary to the strain–stress and strain–damage response given by the trained ANNs, physics principles are considered outside the ANN to handle unloading and reloading scenarios. Tension-compression asymmetry and mesh size dependence are addressed. Examples are shown for plane stress states for which the nonlinear tensorial constitutive relation is captured by a surrogate model, i.e. a trained ANN. The latter is implemented as constitutive material law into a Finite Element Method program to run structural analyses.