An efficient data-driven multiscale framework for modeling anisotropic damage (M-DDHAD) in heterogeneous structures is proposed, where the anisotropic damage model at the macro scale is constructed purely on the knowledge of Representative Volume Elements (RVE) of the material microstructure. The technique involves three main steps: the construction of a database, obtained by performing off-line calculations of crack propagation on Representative Volume Elements (RVE); the construction of an anisotropic damage model constructed from the database using Harmonic Analysis of Damage and off-line calculations, where damage is computed using the constructed model in tandem with a strain-gradient regularization technique. Using Harmonic Analysis of Damage, an anisotropic damage model defining the evolution of the macroscopic elastic tensor as a function of macro internal variables is provided without specific assumptions about the anisotropy related to the RVE geometry. A surrogate model is constructed to define their evolution. The macroscopic problem uses the constructed anisotropic damage model, and a modified strain-gradient regularization is applied to guarantee mesh-independence. The technique accuracy and robustness has been assessed on several structural problems with different microstructures, involving a strong initial and induced anisotropic fracture behavior, and compared with direct crack numerical simulations (DNS) of heterogeneous structures. Very good accuracy has been obtained both regarding the force-displacement curves as well as crack paths, while keeping the efficiency of classical Finite Element simulations.