it.information theory - Closed form Information measure on a dataset

I'm not sure I entirely understand the statement of the question, but it sounds to me as though it's similar to the question, "If you have access to a time series of observations/data coming from some dynamical system, but know nothing about the underlying system itself, what properties of that system can you determine based solely on your observations?"

Some of the key words relating to this question are "correlation dimension" and "correlation entropy", which originate in work of Grassberger and Procaccia ("Measuring the Strangeness of Strange Attractors", Physica D: Nonlinear Phenomena 9, 1983, 189‒208). These let you determine certain properties of the system (analogous to Hausdorff dimension of the attractor or Kolmogorov-Sinai entropy) based only on the time series, without knowing anything about the underlying dynamics, which seems not unrelated to your original question.