Forces between colloidal particles cannot generally be measured directly while they contribute to particle movements and ultimately drive self-assembly. We develop methods to infer the forcefield in a colloidal systems from experimentally accessible trajectories. The motion of colloids is commonly described by the Overdamped Langevin Equation (OLE) for both in- and out-of-equilibrium conditions [1,2,3,4]. According to the OLE, colloidal motion is the sum of two terms: a deterministic motion caused by the total force between the colloid that depends on their (respective) position and a stochastic motion due to the solvent thermal agitation, known as diffusion. Consequently, direct measurement of colloidal motion gives only access to a noisy estimator of the total force and nothing can be said about the different components in the forcefield (i.e external field, pairwise interaction, …). Furthermore, hydrodynamic couplings make the force inference even more complex as it makes the diffusion state dependent while the motion of one colloid becomes correlated with the motion of all the surrounding colloids [3,4,5]. To infer forcefield components from trajectories, it is therefore crucial to infer the hydrodynamic couplings. Following the core idea developed in [6], we introduce a basis with physical inductive biases to capture the different phenomena behind hydrodynamic couplings. Then, using the inferred hydrodynamic couplings, we decorrelate colloidal motions and apply a similarly designed basis to disentangle the different forcefield components. Within this approach, we validate our method on both passive and active 3D systems simulated using brownian dynamics with hydrodynamics [7]. We show that we are able to get very good quantitative agreements between the exact and inferred hydrodynamic couplings as well as the external field and pairwise interactions. References: [1] I. C. Jenkins, Soft Matter, 11, 6948-6956 (2015) [2] G. Volpe, Rep. Prog. Phys., 79, 053901 (2016) [3] G. Volpe, Phys. Rev. Lett. 104, 170602 (2010) [4] G. Pesce, .Phys. Rev. E, 90, 042309 (2014) [5] E. Wajnryb, Journal of Fluid Mechanics 731, 3 (2013) [6] A. Frishman, Phys. Rev. X, 10, 021009 (2020) [7] A. Callegari, In: F. Toschi, M. Sega, (eds) Flowing Matter. Soft and Biological Matter. Springer, Cham. (2019)