Global universal approximation of functional input maps on weighted spaces

Introduction

We introduce a generalization of neural networks to infinite dimensional spaces which we call functional input neural networks (FNN). These neural networks can be applied as supervised learning tools in machine learning (see ), when the input and possibly also the output spaces are infinite dimensional.

In order to show the global universal approximation property of FNNs between infinite dimensional spaces, we use the structure of the weighted spaces and rely on a weighted version of the classical Stone-Weierstrass theorem (see e.g. ) proved in Section 3. Our formulation of this weighted Stone-Weierstrass theorem is inspired by Leopoldo Nachbin’s article , even though our setting differs in several important respects (definition of function spaces, weighted topologies, criteria for density, etc.). The crucial ingredient for global approximation is a weight function that controls the functions to be approximated outside of large compact sets and in turn allows to prove density of point separating moderate growth algebras (introduced in Definition 3.4) among all continuous functions which can be dominated by this weight function. Another Stone-Weierstrass theorem was proved in and for the space of continuous bounded functions over a non-compact space using the so-called strict topology (see also ), which can be seen as the projective limit of the weighted spaces introduced in this paper.

Let us remark that there is of course an extensive literature on neural networks with infinite dimensional input and output spaces. Early instances can be found in for learning a non-linear functional with continuous functions as input. For more recent results we refer to for the universal approximation on non-Euclidean spaces using feature maps and to for the approximation on Fréchet spaces. Moreover, echo-state network architectures were considered in and so-called metric hypertransformers in , which are both shown to be universal for adapted maps between suitably defined discrete-time path spaces. In addition, for learning the solution of a partial differential equation, we refer to the works on physics-informed neural networks in , neural operators in , DeepONets in , and the references therein.

The crucial novelty of the current work are the global UATs for (generalizations of) continuous functions beyond compacts. While classical UATs on compacts ensure the existence of an approximation over a fixed compact set, our results yield a global approximation. Hence, the model can be retrained with a new training set that might be not contained in an a priori fixed compact set. These non-compact UATs are highly relevant in areas like stochastic analysis or mathematical finance, where the model space is given by a set of paths, which is generically non-compact. An important class of functional input maps in mathematical finance are so-called non-anticipative functionals introduced in . We translate our general UAT to this setup and illustrate in Section 7 the numerical performance of FNNs when approximating the running maximum of a standard Brownian motion.

Apart from neural networks, there are many other families which can serve as universal approximators on function spaces. One well-known example is the signature of a path which actually serves as linear regression basis. It plays a central role in rough path theory (see e.g., ) and lately also in econometrics and mathematical finance see e.g., and the references therein). Similarly as for neural networks, UATs for linear functions of the signature have been proved on compact sets of paths (see e.g. for continuous semimartingales and for càdlàg paths). Note however that in the context of (semi-)martingales the Wiener-Ito chaos decomposition with iterated stochastic integrals instead of the signature (see for Brownian motion and for processes with stationary increments) can be seen as global UAT for the $L^{2}$-norm with respect to the Wiener measure (see also the $L^{p}$-version of for diffusion processes and for càdlàg semimartingales).

By choosing an appropriate weight function and by applying the weighted Stone-Weierstrass theorem we here obtain a global universal approximation result for linear functions of the signature of continuous geometric rough paths (see Section 5). Let us remark that in also a global approximation result is obtained, however not with the “true” signature but with a bounded normalization, as there the strict topology from on continuous bounded functions (see [23, Defintion 8]) is used. The disadvantage of this normalization is that the tractability properties of the signature, like computing its expected value analytically, are lost. In Section 6 we also introduce the viewpoint of Gaussian process regression in this setting and show that the reproducing kernel Hilbert space of the “true” signature kernels are Cameron-Martin spaces of certain Gaussian processes. This paves the way towards uncertainty quantification for signature kernel regression.

In the following subsection we provide three specific examples of FNNs that can be used to approximate $\alpha$-Hölder continuous functions, non-anticipative paths functionals as well as monetary risk measures.

In order to give an overview of the applicability of our results, we consider three particular examples. The first one is within functional data analysis, which is a branch of statistics where every sample corresponds to a continuous function (see ). Hence, we learn a continuous map $f:C^{\alpha}(S)\rightarrow Y$ with some growth conditions, where $(S,d_{S})$ is a compact metric space, $C^{\alpha}(S)$ denotes the space of $\alpha$-Hölder continuous real-valued functions on $(S,d_{S})$, and $(Y,\|\cdot\|_{Y})$ is a Banach space. Then, our results show that the map $f:C^{\alpha}(S)\rightarrow Y$ can be approximated by learning a FNN $\varphi:C^{\alpha}(S)\rightarrow Y$ of the form

These three examples provide a first glance over the broad range of applications for FNNs. Note that the input space can violate the vector space structure, as illustrated by the second example, and can be the dual of another Banach space, as in the third example.

The remainder of the article is structured as follows. In the next subsection we introduce relevant notation used throughout the article. Section 2 is dedicated to notions related to weighted spaces and the generalization of continuous functions, called $\mathcal{B}_{\psi}$-functions, defined thereon. In Section 3, we prove the weighted Stone-Weierstrass theorem in this setting, which we then use in Section 4 to lift the global universal approximation property of classical neural networks to FNNs. In Section 5, we present a further application of the weighted Stone-Weierstrass theorem to prove a global UAT for linear functions of path signatures. Section 6 then introduces a Gaussian process regression point of view and identifies the reproducing kernel Hilbert space of the signature kernel with the Cameron-Martin space of Gaussian processes taking values in $\mathcal{B}_{\psi}$-functions. Finally, we provide some numerical examples in Section 7.

2 Notation

Furthermore, for a Hausdorff topological space $(X,\tau_{X})$ and a Banach space $(Y,\|\cdot\|_{Y})$, we denote by $C^{0}(X;Y)$ the vector space of continuous maps $f:X\rightarrow Y$, whereas $C^{0}_{b}(X;Y)\subseteq C^{0}(X;Y)$ denotes the vector subspace of bounded maps. If $(X,\tau_{X})$ is additionally compact, every map $f\in C^{0}(X;Y)$ is bounded. In this case, the supremum norm $\|f\|_{C^{0}(X;Y)}:=\sup_{x\in X}\|f(x)\|_{Y}$ turns $C^{0}(X;Y)$ into a Banach space.

Moreover, if $(S,d_{S})$ is a metric space and $(Z,\|\cdot\|_{Z})$ is a Banach space, a subset $A\subseteq C^{0}(S;Z)$ is called equicontinuous if for every $\varepsilon>0$ there exists some $\delta>0$ such that for every $f\in A$ and $s,t\in S$ with $d_{S}(s,t)<\delta$ it holds that $\|f(s)-f(t)\|_{Z}<\varepsilon$. In addition, a subset $A\subseteq C^{0}(S;Z)$ is called pointwise bounded (resp. pointwise compact) if for every $s\in S$ the set $\{f(s):f\in A\}$ is bounded (resp. compact) in $Z$.

One of our most important example for a weighted space $X$, as introduced in Section 2 below, are Hölder spaces defined as follows. For some $\alpha\in(0,1]$, a compact metric space $(S,d_{S})$ with designated origin $0\in S$, and a dual space $Z$ equipped with the weak-$*$-topology, we denote by $C^{\alpha}(S;Z)$ the space of $\alpha$-Hölder continuous functions $x:S\rightarrow Z$ with finite norm

where $\|\cdot\|_{\alpha}$ denotes the $\alpha$-Hölder seminorm of $x:S\rightarrow Z$ defined as

Unless otherwise specified, we endow $C^{\alpha}(S;Z)$ with the $\alpha$-Hölder norm $\|\cdot\|_{C^{\alpha}(S;Z)}$, which turns $C^{\alpha}(S;Z)$ into a Banach space (see [47, Theorem 5.25 (ii)] and [102, Proposition 2.3(b)]). For $\alpha=1$, we can relate $C^{\alpha}(S;Z)$ to the notion of globally Lipschitz continuous functions $x:S\rightarrow Z$ considered in . Moreover, we denote by $C^{\alpha}_{0}(S;Z)\subseteq C^{\alpha}(S;Z)$ the closed vector subspace of $\alpha$-Hölder continuous functions $x\in C^{\alpha}(S;Z)$ preserving the origin, i.e. $x(0)=0\in Z$.

The second important example are spaces of finite $p$-variation. For some $T>0$, $p\geq 1$, and a dual space $Z$ equipped with the weak-$*$-topology, we denote by $C^{p-var}([0,T];Z)$ the space of continuous paths $x:[0,T]\rightarrow Z$ with finite norm

Hereby, $\|x\|_{p-var}$ denotes the $p$-variation of $x:[0,T]\rightarrow Z$ defined as

We shall also need to intersect spaces of finite $p$-variation with Hölder spaces. For some $T>0$, $(p,\alpha)\in[1,\infty)\times(0,1)$ with $p\alpha<1$, and a dual space $Z$ equipped with the weak-$*$-topology, we define the intersection $C^{p-var,\alpha}([0,T];Z):=C^{p-var}([0,T];Z)\cap C^{\alpha}([0,T];Z)$ which consists of $\alpha$-Hölder continuous paths $x:[0,T]\rightarrow Z$ with finite $p$-variation. Unless otherwise specified, we endow $C^{p-var,\alpha}([0,T];Z)$ with the norm

For the weighted space setting it will be essential to equip the Hölder spaces $C^{\alpha}(S;Z)$ and the spaces $C^{p-var,\alpha}(S;Z)$ also with weaker topologies than the norm topology, for example, with the uniform topology induced by $\|\cdot\|_{C^{0}(S;Z)}$ or a weak-$*$-topology, which is precisely defined in Appendix A.

Weighted spaces and functions thereon

For the universal approximation results as well as the weighted Stone-Weierstrass theorems on infinite dimensional spaces, we use a weighted space as input space and a Banach space as output space. On the weighted input space, one can in turn introduce a weighted function space, which was under (slightly) different conditions also studied in .

Our weighted setting is in particular inspired by , where weighted spaces have been used for Kolmogorov equations, splitting schemes of (stochastic) partial differential equations, and generalized Feller processes. To make the current article self-contained, we shall recall all necessary definitions and provide several examples.

In order to define a weighted space we shall assume throughout that $(X,\tau_{X})$ is a completely regular Hausdorff topological space.

A function $\psi:X\rightarrow(0,\infty)$ is called an admissible weight function (on $(X,\tau_{X})$) if every pre-image $K_{R}:=\psi^{-1}((0,R])=\left\{x\in X:\psi(x)\leq R\right\}$ is compact with respect to $\tau_{X}$, for all $R>0$.

The definition of a weighted space has the following important consequences.

If $(X,\tau_{X})$ is a separable locally convex topological vector space and $\psi:X\rightarrow(0,\infty)$ is convex, then $\psi:X\rightarrow(0,\infty)$ is continuous on $E\subseteq X$ if and only if $E$ is locally compact, see [36, Remark 2.2].

Note that by Baire’s category theorem $(X,\tau_{X})$ can only be a Banach space if it is finite-dimensional.

One of our main examples for a weighted space (see Example 2.3 (ii) below) are spaces $X$ which are the dual of some Banach space, equipped with a weak-$*$-topology. Again, by Baire’s category theorem, these spaces are metrizable if and only if they are finite dimensional. Moreover, in infinite dimensions completeness with respect to the weak-$*$-topology also fails, since the completion would consist of all linear functions and not only of continuous ones.

In the following, we present some examples of weighted spaces $(X,\psi)$, where the compactness of the pre-images $K_{R}=\psi^{-1}((0,R])$ needs to be verified with a suitable criterion (e.g. the Banach-Alaoglu theorem or the Arzelà-Ascoli theorem).

The examples of admissible weight functions $\psi:X\rightarrow(0,\infty)$ that we present in the following are all of the form $\psi(x)=\eta\left(\|x\|\right)$ for a norm $\|\cdot\|$ on $X$ and a continuous increasing function $\eta:[0,\infty)\rightarrow(0,\infty)$. In view of the so-called moderate growth condition defined below (see Definition 3.4), we consider in particular the continuous increasing function $\eta(r)=\exp\left(\beta r^{\gamma}\right)$, for some $\beta>0$ and $\gamma>1$.

If $(X,\|\cdot\|_{X})$ is a dual space equipped with the weak-$*$-topology, i.e. if there exists a Banach space $(E,\|\cdot\|_{E})$ and an isometric isomorphism $\Phi:X\rightarrow E^{*}$, we choose the weight function $\psi(x)=\eta\left(\|x\|_{X}\right)$. Then, the pre-image $K_{R}=\psi^{-1}((0,R])$ is by the Banach-Alaoglu theorem compact in the weak-$*$-topology, for all $R>0$. This shows that $(X,\psi)$ is a weighted space.

For $\alpha>0$, a compact metric space $(S,d_{S})$, and a dual space $Z$ equipped with the weak-$*$-topology, let $X=C^{\alpha}(S;Z)$ be the space of $\alpha$-Hölder continuous functions $x:S\rightarrow Z$ introduced in Section 1.2. Then, $C^{\alpha}(S;Z)$ is by Theorem A.4 again a dual space, which means that $C^{\alpha}(S;Z)$ can be equipped with a weak-$*$-topology. In this case, $\psi(x)=\eta\left(\|x\|_{C^{\alpha}(S;Z)}\right)$ is by the Banach-Alaoglu theorem admissible, which shows that $(C^{\alpha}(S;Z),\psi)$ is a weighted space. Similarly, if $(S,d_{S})$ is additionally pointed, we can consider functions in $C^{\alpha}_{0}(S;Z)$ preserving the origin (see Section 1.2), where the same reasoning applies (see Remark A.7 (ii)).

Opposed to (iii), let $X=C^{\alpha}(S;Z)$ be the space of $\alpha$-Hölder continuous functions $x:S\rightarrow Z$, but now equipped with the supremum norm $\|\cdot\|_{C^{0}(S;Z)}$. Then, we choose $\psi(x)=\eta\left(\|x\|_{C^{\alpha}(S;Z)}\right)$ and observe that the pre-image $K_{R}=\psi^{-1}((0,R])$ is equicontinuous and pointwise bounded, thus by the Banach-Alaoglu theorem pointwise compact with respect to the weak-$*$-topology of $Z$. Hence, $K_{R}=\psi^{-1}((0,R])$ is by the Arzelà-Ascoli theorem in [59, Theorem 7.17] compact with respect to $\|\cdot\|_{C^{0}(S;Z)}$, for all $R>0$, which shows that $(C^{\alpha}(S;Z),\psi)$ is a weighted space. Note that instead of the uniform topology we could also use any $C^{\alpha^{\prime}}$-topology for $0<\alpha^{\prime}<\alpha$ due to the compact embedding $C^{\alpha}(S;Z)\hookrightarrow C^{\alpha^{\prime}}(S;Z)$ (see Theorem A.3).

For $\alpha>0$ and $T>0$, let $X=\Lambda^{\alpha}_{T}$ be the space of stopped $\alpha$-Hölder continuous paths (as introduced in Section 1.1), i.e.

For $T>0$, $(p,\alpha)\in[1,\infty)\times(0,1)$ with $p\alpha<1$, and a dual space $Z$ equipped with the weak-$*$-topology, let $X=C^{p-var,\alpha}([0,T];Z)$ be the space of $\alpha$-Hölder continuous paths $x:[0,T]\rightarrow Z$ with finite $p$-variation (see Section 1.2). Then, $C^{p-var,\alpha}([0,T];Z)$ is by Theorem A.9 again a dual space and can be equipped with a weak-$*$-topology. Hence, $\psi(x)=\eta\left(\|x\|_{C^{p-var,\alpha}([0,T];Z)}\right)$ is by the Banach-Alaoglu theorem admissible, which shows that $(C^{p-var,\alpha}([0,T];Z),\psi)$ is a weighted space. Similarly, we can consider paths in $C^{p-var,\alpha}_{0}([0,T];Z)$ preserving the origin (see Section 1.2), where the same reasoning applies (see Remark A.12).

Opposed to (vi), let $X=C^{p-var,\alpha}([0,T];Z)$ be the space of $\alpha$-Hölder continuous paths $x:[0,T]\rightarrow Z$ with finite $p$-variation, but now equipped with the supremum norm $\|\cdot\|_{C^{0}([0,T];Z)}$. Then, we choose $\psi(x)=\eta\left(\|x\|_{C^{p-var,\alpha}([0,T];Z)}\right)$ and use the compact embedding $C^{p-var,\alpha}([0,T];Z)\hookrightarrow C^{0}([0,T];Z)$ in Theorem A.8 to conclude that $K_{R}=\psi^{-1}((0,R])$ is compact with respect to $\|\cdot\|_{C^{0}([0,T];Z)}$, for all $R>0$. Hence, $(C^{p-var,\alpha}([0,T];Z),\psi)$ is a weighted space. Note that instead of the uniform topology we could also use any $C^{p^{\prime}-var,\alpha^{\prime}}$-topology for $(p^{\prime},\alpha^{\prime})\in(p,\infty]\times[0,\alpha)$ due to the compact embedding $C^{p-var,\alpha}([0,T];Z)\hookrightarrow C^{p^{\prime}-var,\alpha^{\prime}}([0,T];Z)$ (see Theorem A.8).

The above examples give rise to the following observations:

Note that in all the infinite dimensional examples (ii) to (ix) the idea is always to use a weaker topology than the norm topology which renders the closed unit ball compact. In the above cases this is either achieved with the weak-$*$-topology as in Example (ii), (iii), (vi), (viii), and (x), or via the following compact embeddings:

$C^{\alpha}(S;Z)\hookrightarrow C^{\alpha^{\prime}}(S;Z)$ for $0\leq\alpha^{\prime}<\alpha$ as in Example (iv) and (v),

$C^{p-var,\alpha}([0,T];Z)\hookrightarrow C^{p^{\prime}-var,\alpha^{\prime}}([0,T];Z)$ for $(p,\alpha)\in[1,\infty)\times(0,1)$ with $p\alpha<1$ and $(p^{\prime},\alpha^{\prime})\in(p,\infty]\times[0,\alpha)$ with $p^{\prime}\alpha^{\prime}<1$ as in Example (vii), where $C^{\infty-var,0}([0,T];Z):=C^{0}([0,T];Z)$, $C^{p^{\prime}-var,0}([0,T];Z):=C^{p^{\prime}-var}([0,T];Z)$, and $C^{\infty-var,\alpha^{\prime}}([0,T];Z):=C^{\alpha^{\prime}}([0,T];Z)$.

$L^{p}$-embeddings of Sobolev–Slobodeckij spaces (see e.g. ).

Consider the setting of Example 2.3 (iii). Then, as shown in Theorem A.4, the weak-$*$-topology and the $C^{\alpha^{\prime}}(S;Z)$-topologies for $0\leq\alpha^{\prime}<\alpha$ (including the uniform topology with $\alpha^{\prime}=0$) coincide on any $\|\cdot\|_{C^{\alpha}(S;Z)}$-bounded and sequentially weak-$*$-closed subset of $C^{\alpha}(S;Z)$, but are globally clearly different. The same reasoning applies to Example 2.3 (vi), see Theorem A.9.

For a given weighted space $(X,\psi)$ with admissible weight function $\psi:X\rightarrow(0,\infty)$ and a Banach space $(Y,\|\cdot\|)$, we need to introduce an appropriate weighted function space that can be used for the subsequent global approximation theorems. Besides the vector space of bounded and continuous maps $C^{0}_{b}(X;Y)$, we define the vector space

of maps $f:X\rightarrow Y$, whose growth is controlled by the growth of the weight function $\psi:X\rightarrow(0,\infty)$. We equip $B_{\psi}(X;Y)$ with the weighted norm $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$ given by

for $f\in B_{\psi}(X;Y)$. Then, the continuous embedding $C^{0}_{b}(X;Y)\hookrightarrow B_{\psi}(X;Y)$ holds true.

Note that if $X$ is compact, the weight function $\psi(x)=1$ is admissible. However, on general spaces, $\psi:X\rightarrow(0,\infty)$ grows on the compact pre-images $K_{R}:=\psi^{-1}((0,R])$, which means that the elements of $\mathcal{B}_{\psi}(X;Y)$ are typically unbounded, but their growth is controlled by the growth of the weight function $\psi:X\rightarrow(0,\infty)$.

For simplicity, we always assume that the output space is a Banach space $(Y,\|\cdot\|_{Y})$. However, all the following results (including the weighted vector-valued Stone-Weierstrass theorem in Theorem 3.8) still hold true with a locally convex topological vector space $(Y,\tau_{Y})$ as output space. In this case, the topology on $Y$ and $\mathcal{B}_{\psi}(X;Y)$ is generated by families of seminorms instead of $\|\cdot\|_{Y}$ and $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$, respectively (see ).

The following holds true for a function $f:X\rightarrow Y$:

If $f\in\mathcal{B}_{\psi}(X;Y)$ then $f|_{K_{R}}\in C^{0}(K_{R};Y)$, for all $R>0$, and

then $f\in\mathcal{B}_{\psi}(X)$. In particular, $f\in\mathcal{B}_{\psi}(X)$ for every $f\in C^{0}(X)$ satisfying (2.3).

For (i), let $f\in\mathcal{B}_{\psi}(X;Y)$ and fix some $\varepsilon>0$. Then, there exists by definition of $\mathcal{B}_{\psi}(X;Y)$ some $g\in C^{0}_{b}(X;Y)$ such that $\|f-g\|_{\mathcal{B}_{\psi}(X;Y)}<\varepsilon/2$. Hence, by choosing $R\geq 2\varepsilon^{-1}\sup_{x\in X}\|g(x)\|_{Y}$, it follows for every $x\in X\setminus K_{R}$ that

Since $\varepsilon>0$ was chosen arbitrarily, we obtain (2.2). Moreover, with $g\in C^{0}_{b}(X;Y)$ from above, we observe for every $R>0$ that

This shows that $f|_{K_{R}}\in C^{0}(K_{R};Y)$ is continuous as uniform limit of continuous functions, for all $R>0$. On the other hand, (ii) follows from [40, Theorem 2.7]. ∎

In addition, by following [40, Theorem 2.8], there exists for every $f\in\mathcal{B}_{\psi}(X)$ with $\sup_{x\in X}f(x)>0$ some $x_{0}\in X$ such that $f(x)/\psi(x)\leq f(x_{0})/\psi(x_{0})$, for all $x\in X$, which shows the analogy to functions vanishing at infinity on locally compact spaces.

Weighted Stone-Weierstrass theorems

In this section, we generalize the Stone-Weierstrass theorems to this weighted setting with non-compact input space. For this purpose, we shortly recall the classical Stone-Weierstrass theorems formulated for compact domains, which we will need for our results.

Subsequently, the Weierstrass approximation result in Theorem 3.1 was generalized by Marshall Harvey Stone in to more general spaces by using the notion of a subalgebra. Hereby, a vector subspace $\mathcal{A}\subseteq C^{0}(X)$ is called a subalgebra (of $C^{0}(X)$) if $\mathcal{A}$ is closed under multiplication, i.e. for every $a_{1},a_{2}\in\mathcal{A}$ it holds that $a_{1}\cdot a_{2}\in\mathcal{A}$.

Let $(X,\tau_{X})$ be a compact Hausdorff topological space and assume that $\mathcal{A}\subseteq C^{0}(X)$ is a subalgebra. Then, $\mathcal{A}$ is dense in $C^{0}(X)$ if and only if $\mathcal{A}$ is point separating and vanishes nowhere.

Next, we state the vector-valued Stone-Weierstrass of R. Creighton Buck in . For a given subalgebra $\mathcal{A}\subseteq C^{0}(X)$, a vector subspace $\mathcal{W}\subseteq C^{0}(X;Y)$ is called an $\mathcal{A}$-submodule if $a\cdot w\in\mathcal{W}$, for all $a\in\mathcal{A}$ and $w\in\mathcal{W}$, where $x\mapsto(a\cdot w)(x):=a(x)w(x)$. For further details on vector-valued approximation, we refer to the textbook .

Let $(X,\tau_{X})$ be a compact Hausdorff space, let $\mathcal{A}\subseteq C^{0}(X)$ be a subalgebra and let $\mathcal{W}\subseteq C^{0}(X;Y)$ be an $\mathcal{A}$-submodule. Then, the following are equivalent.

$\mathcal{A}$ is point separating and vanishes nowhere, and $\mathcal{W}(x):=\{w(x):w\in\mathcal{W}\}$ is dense in $Y$, for all $x\in X$.

Later, the vector-valued Stone-Weierstrass was extended by Erret Bishop in to a measure theoretic version. In addition, Silvio Machado derived in a quantitative version, which relies on Zorn’s lemma, see also the proofs of and .

2 Weighted real-valued Stone-Weierstrass theorem

Note that Nachbin’s setting in differs in several respects, in particular in the definition of weighted topologies. It allows for multiple upper semicontinuous weights $v:X\rightarrow[0,\infty)$, e.g. $v(x)=\psi(x)^{-1}$, collected in a so-called Nachbin family and can therefore represent the compact-open topology, the strict topology, and other topologies.

For a given weight function $\psi:X\rightarrow(0,\infty)$, a subalgebra $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ is called point separating of $\psi$-moderate growth if there exists a point separating vector subspace $\widetilde{\mathcal{A}}\subseteq\mathcal{A}$ such that $x\mapsto\exp\left(\left|\widetilde{a}(x)\right|\right)\in\mathcal{B}_{\psi}(X)$, for all $\widetilde{a}\in\widetilde{\mathcal{A}}$.

A subalgebra $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ is point separating of $\psi$-moderate growth under any given admissible weight function $\psi:X\rightarrow(0,\infty)$ if the subalgebra $\mathcal{A}$ is point separating and consists only of bounded maps. This is related to the so-called “bounded approximation problem” of Nachbin, see [79, Theorem 1].

Let $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ be a subalgebra that is point separating of $\psi$-moderate growth and vanishes nowhere. Then, $\mathcal{A}$ is dense in $\mathcal{B}_{\psi}(X)$.

It suffices to show the approximation of a map $f\in C^{0}_{b}(X)$ by an element $a\in\mathcal{A}$ since $C^{0}_{b}(X)$ is by definition dense in $\mathcal{B}_{\psi}(X)$. We first assume that $\mathcal{A}$ consists only of bounded maps, where the condition $\mathcal{A}$ being point separating of $\psi$-moderate growth reduces to $\mathcal{A}$ being point separating. Let $f\in C^{0}_{b}(X)$, fix some $\varepsilon>0$, and define the constants $M=\left(\inf_{x\in X}\psi(x)\right)^{-1}>0$ as well as $b=\sup_{x\in X}|f(x)|+\varepsilon/(4M)$ and $R\geq 4b/\varepsilon$. Observe that the restriction $\mathcal{A}|_{K_{R}}$ is a point separating subalgebra of $C^{0}(K_{R})$, which vanishes nowhere. Hence, by using the classical real-valued Stone-Weierstrass in Theorem 3.2, there exists some $a\in\mathcal{A}$ such that

Hence, by combining (3.1) with (3.2), we conclude that

Since $\mathcal{A}$ is a subalgebra, i.e. $p\circ a\in\mathcal{A}$, and $\varepsilon>0$ as well as $f\in C^{0}_{b}(X)$ were chosen arbitrarily, it follows that $\mathcal{A}\subseteq C^{0}_{b}(X)$ is dense in $\mathcal{B}_{\psi}(X)$.

For the general case of a point separating subalgebra $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ of $\psi$-moderate growth, with point separating vector subspace $\widetilde{\mathcal{A}}\subseteq\mathcal{A}$ such that $x\mapsto\exp\left(\left|\widetilde{a}(x)\right|\right)\in\mathcal{B}_{\psi}(X)$, for all $\widetilde{a}\in\widetilde{\mathcal{A}}$, we first show that the maps $x\mapsto\cos\left(\widetilde{a}(x)\right)$ and $x\mapsto\sin\left(\widetilde{a}(x)\right)$, with $\widetilde{a}\in\widetilde{\mathcal{A}}$, belong to the $\|\cdot\|_{\mathcal{B}_{\psi}(X)}$-closure of $\mathcal{A}$. Indeed, let $\widetilde{a}\in\widetilde{\mathcal{A}}$ and fix some $\varepsilon>0$. Then, by applying Lemma 2.7 (i), there exists some $R>4/\varepsilon$ such that

Since $\varepsilon>0$ was chosen arbitrarily, it follows that $x\mapsto\cos\left(\widetilde{a}(x)\right)$ belongs to the $\|\cdot\|_{\mathcal{B}_{\psi}(X)}$-closure of $\mathcal{A}$, which holds analogously true for $x\mapsto\sin\left(\widetilde{a}(x)\right)$. Thus, the subalgebra

of $\mathcal{B}_{\psi}(X)$ is contained in the $\|\cdot\|_{\mathcal{B}_{\psi}(X)}$-closure of $\mathcal{A}$. Hence, by applying the previous step to the point separating subalgebra $\mathcal{A}_{\text{trig}}\subseteq\mathcal{B}_{\psi}(X)$ which vanishes nowhere and consists of bounded maps, we conclude that $\mathcal{A}_{\text{trig}}$ is dense in $\mathcal{B}_{\psi}(X)$. However, since $\mathcal{A}_{\text{trig}}$ is contained in the $\|\cdot\|_{\mathcal{B}_{\psi}(X)}$-closure of $\mathcal{A}$, it follows that $\mathcal{A}$ is also dense in $\mathcal{B}_{\psi}(X)$. ∎

3 Weighted vector-valued Stone-Weierstrass theorem

In this section, we generalize the weighted Stone-Weierstrass to the vector-valued case, where $(Y,\|\cdot\|_{Y})$ is a Banach space. For this purpose, we introduce for every $w\in\mathcal{B}_{\psi}(X;Y)$ the tilted weight function $\psi_{w}:X\rightarrow(0,\infty)$ defined by $\psi_{w}(x)=\frac{\psi(x)}{1+\|w(x)\|_{Y}}$, for $x\in X$.

For every $w\in\mathcal{B}_{\psi}(X;Y)$, the weight function $\psi_{w}:X\rightarrow(0,\infty)$ is admissible.

We fix some $R>0$. Clearly the set $K_{w,R}:=\left\{x\in X:\psi_{w}(x)\leq R\right\}$ lies in a compact set of the form $K_{C}:=\left\{x\in X:\psi(x)\leq C\right\}$ for some constant $C>0$. A priori it is, however, not clear that $K_{w,R}$ is compact. It is of course sufficient to show that it is closed, since it lies in the compact set $K_{C}$. By Lemma 2.7 (i) we know that \|w\|_{Y}\big{|}_{K} is continuous, whence \left(\|w\|_{Y}\big{|}_{K}\right)^{-1}([a,b]) is closed. Therefore

Let $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ be a subalgebra that vanishes nowhere and is point separating of $\psi_{w}$-moderate growth, for all $w\in\mathcal{W}$, where $\mathcal{W}\subseteq\mathcal{B}_{\psi}(X;Y)$ is an $\mathcal{A}$-submodule such that $\mathcal{W}(x):=\{w(x):w\in\mathcal{W}\}$ is dense in $Y$, for all $x\in X$. Then, $\mathcal{W}$ is dense in $\mathcal{B}_{\psi}(X;Y)$.

Let $\mathcal{B}_{b}(X)\subseteq\mathcal{B}_{\psi}(X)$ be the vector subspace of bounded maps. Then, we first show that the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{W}$ is a $\mathcal{B}_{b}(X)$-submodule. For this purpose, we fix some $g\in\mathcal{B}_{b}(X)$ and assume that $\overline{w}:X\rightarrow Y$ belongs to the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{W}$. Moreover, we fix some $\varepsilon>0$ and define $c=\sup_{x\in X}|g(x)|<\infty$. Then, there exists some $w\in\mathcal{W}$ with $\|\overline{w}-w\|_{\mathcal{B}_{\psi}(X;Y)}<\varepsilon/(2c)$, which implies that

Now, we observe that the tilted weight function $\psi_{w}:X\rightarrow(0,\infty)$ is by Lemma 3.7 admissible. Moreover, for every $a\in\mathcal{A}$, we have $a\in\mathcal{B}_{\psi}(X)$ and $a\cdot w\in\mathcal{W}\subseteq\mathcal{B}_{\psi}(X;Y)$ as $\mathcal{W}\subseteq\mathcal{B}_{\psi}(X;Y)$ is an $\mathcal{A}$-submodule. Then, Lemma 2.7 (i) implies that

and that $a|_{K_{R}}\in C^{0}(K_{R})$, for all $R>0$. Thus, by using Lemma 2.7 (ii), we have $a\in\mathcal{B}_{\psi_{w}}(X)$, which shows that $\mathcal{A}$ is also a subalgebra of $\mathcal{B}_{\psi_{w}}(X)$ as $a\in\mathcal{A}$ was chosen arbitrarily. Hence, by applying the weighted real-valued Stone-Weierstrass in Theorem 3.6 on $\mathcal{B}_{\psi_{w}}(X)$, there exists some $a\in\mathcal{A}$ with $\|g-a\|_{\mathcal{B}_{\psi_{w}}(X)}<\varepsilon/2$, which implies that

By combining (3.4) with (3.5), we conclude that

Since $\mathcal{W}$ is an $\mathcal{A}$-submodule, i.e. $a\cdot w\in\mathcal{W}$, and $\varepsilon>0$ was chosen arbitrarily, it follows that $g\cdot\overline{w}:X\rightarrow Y$ belongs to the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{W}$. By using that $g\in\mathcal{B}_{b}(X)$ and $w\in\mathcal{W}$ were also chosen arbitrarily, the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{W}$ is a $\mathcal{B}_{b}(X)$-submodule.

Finally, it suffices to prove the approximation of some $f\in C^{0}_{b}(X;Y)$ by an element $w\in\mathcal{W}$ since $C^{0}_{b}(X;Y)$ is by definition dense in $\mathcal{B}_{\psi}(X;Y)$. Let $f\in C^{0}_{b}(X;Y)$, fix some $\varepsilon>0$, define $b=\sup_{x\in X}\|f(x)\|_{Y}$ as well as $M=\max\left(1,\left(\inf_{x\in X}\psi(x)\right)^{-1}\right)$ and choose $R>\max(4(b+1)/\varepsilon,1)$. We observe that the restricted subalgebra $\mathcal{A}|_{K_{R}}$ of $C^{0}(K_{R})$ and the restricted $\mathcal{A}|_{K_{R}}$-submodule $\mathcal{W}|_{K_{R}}$ of $C^{0}(K_{R};Y)$ satisfy the assumptions of the classical vector-valued Stone-Weierstrass in Theorem 3.3 (i). Hence, there exists some $w\in\mathcal{W}$ such that

Now, for $c=\sup_{x\in K_{R}}\|w(x)\|_{Y}$, it follows from (3.6) that $c\leq\varepsilon/(4M)+b$. Moreover, we define $g\in\mathcal{B}_{b}(X)$ by $g(x)=\min(1+c,1+\|w(x)\|_{Y})/(1+\|w(x)\|_{Y})$, for $x\in X$, which implies that $g(x)=1$, for all $x\in K_{R}$. Then, $g\cdot w:X\rightarrow Y$ is bounded with $\sup_{x\in X}\|g(x)w(x)\|_{Y}\leq 1+c\leq 1+b+\varepsilon/(4M)$. Hence, (3.6) and $M,R\geq 1$ show that

Since the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{W}$ is by the previous step a $\mathcal{B}_{b}(X)$-submodule, i.e. $g\cdot w:X\rightarrow Y$ belongs to the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{W}$, and $\varepsilon>0$ as well as $f\in C^{0}_{b}(X;Y)$ were chosen arbitrarily, it follows that $\mathcal{W}$ is dense in $\mathcal{B}_{\psi}(X;Y)$. ∎

Theorem 3.8 can be generalized to the setting with a locally convex topological vector space $(Y,\tau_{Y})$ as output space (see and Remark 2.6).

In Theorem 3.8, $\mathcal{A}\subset\mathcal{B}_{\psi}(X)$ is point separating of $\psi_{w}$-moderate growth for all $w\in\mathcal{W}$, if either $\mathcal{A}\subset\mathcal{B}_{\psi}(X)$ consists only of bounded functions (see Remark 3.5) or if $\mathcal{A}\subset\mathcal{B}_{\sqrt{\psi}}(X)$ is of $\sqrt{\psi}$-moderate growth and $\mathcal{W}$ is a subset of $\mathcal{B}_{\sqrt{\psi}}(X;Y)$. In both cases, the multiplication $\mathcal{A}\times\mathcal{W}\ni(a,w)\mapsto a\cdot w\in\mathcal{B}_{\psi}(X;Y)$ becomes jointly continuous.

The vector-valued Stone-Weierstrass in Theorem 3.8 is a version of Nachbin’s weighted approximation result in [79, Theorem 6]. Moreover, João B. Prolla extended in the measure theoretic version of Erret Bishop in to weighted spaces, see also .

Universal approximation on weighted spaces

First, we introduce the infinite dimensional analogue of affine maps, which are applied in classical neural networks on the hidden layer.

A subset $\mathcal{H}\subseteq\mathcal{B}_{\psi}(X)$ is called an additive family (on $X$) if

$\mathcal{H}$ is closed under addition, i.e. for every $h_{1},h_{2}\in\mathcal{H}$ it holds that $h_{1}+h_{2}\in\mathcal{H}$,

$\mathcal{H}$ is point separating, i.e. for distinct $x_{1},x_{2}\in X$ there is $h\in\mathcal{H}$ with $h(x_{1})\neq h(x_{2})$.

We now give some examples of additive families. If the weighted space $(X,\psi)$ is a vector space equipped with a norm, then we can use the dual space of its completion.

Hence, Lemma 2.7 (ii) shows that $h\in\mathcal{B}_{\psi}(X)$ and thus $\mathcal{H}\subseteq\mathcal{B}_{\psi}(X)$.

If $(X,\|\cdot\|_{0})$ is a Banach space in Lemma 4.9 such that $\psi:X\rightarrow(0,\infty)$ is admissible, then $X$ is by Remark 2.2 (ii) finite dimensional. Hence, for an infinite dimensional weighted space $(X,\psi)$, we consider in fact an incomplete normed vector space $(X,\|\cdot\|_{0})$, where the dual of the $\|\cdot\|_{0}$-completion is used for the additive family.

Lemma 4.9 shows that a FNN $\varphi:X\rightarrow Y$ over a weighted normed vector space $(X,\psi)$ can be chosen of the form

On the other hand, if the weighted space $(X,\psi)$ is a dual space equipped with the weak-$*$-topology as in Example 2.3 (ii), we obtain the following result.

Hence, Lemma 2.7 (ii) shows that $h\in\mathcal{B}_{\psi}(X)$ and thus $\mathcal{H}\subseteq\mathcal{B}_{\psi}(X)$.

Lemma 4.11 shows that a FNN $\varphi:X\rightarrow Y$ over a weighted dual space $(X,\psi)$ with isometric isomorphism $\Phi:X\rightarrow E^{*}$ and equipped with the weak-$*$-topology is given by

Let us give some examples of functional input neural networks, where we revisit among others the examples from Section 1.1. Hereby, $\eta:[0,\infty)\rightarrow(0,\infty)$ is a continuous increasing function satisfying $\lim_{r\rightarrow\infty}\frac{r}{\eta(r)}=0$.

For the space of $\alpha$-Hölder continuous functions $X=C^{\alpha}(S)$ as in Example 2.3 (iv) with weight function $\psi(x)=\eta\left(\|x\|_{C^{\alpha}(S)}\right)$, an additive family is given by

For the space of signed Radon measures $X=\mathcal{M}_{\psi_{\Omega}}(\Omega)$ on a weighted space $(\Omega,\psi_{\Omega})$ as in Example 2.3 (x) with weight function $\psi(x)=\eta\left(\|x\|_{\mathcal{M}_{\psi_{\Omega}}(\Omega)}\right)$, an additive family is given by

This can be used to learn a map with a (probability) measure as input.

These results give an overview of the broad variety of functional input neural networks, for which we show the universal approximation property in the following subsection.

2 Global universal approximation

Universal approximation results were first proved for neural networks between Euclidean spaces in and in using functional analytical arguments such as the Hahn-Banach theorem and Fourier theory. Later, further universal approximation results were found for different types of activation functions, such as in , while the results of e.g. , and are concerned with proving quantitative approximation rates under more restrictive assumptions on the involved functions.

Now, for the function $x\mapsto w(x):=\cos(h(x))y_{0}\in\mathcal{W}$, we define the corresponding FNN $x\mapsto\varphi(x):=y_{0}\rho(h(x))\in\mathcal{NN}^{\mathcal{H},\rho,\mathcal{L}}_{X,Y}$. Then, we conclude from (4.2) that

Since $\varepsilon>0$ was chosen arbitrarily, it follows that the map $x\mapsto w(x)=\cos(h(x))y_{0}\in\mathcal{W}$ belongs to the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{NN}^{\mathcal{H},\rho,\mathcal{L}}_{X,Y}$, which holds analogously true for the map $x\mapsto\sin(h(x))y_{0}\in\mathcal{W}$. Hence, we conclude by the triangle inequality that linear combinations of these maps belong to the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{NN}^{\mathcal{H},\rho,\mathcal{L}}_{X,Y}$, which shows that the entire $\mathcal{A}$-submodule $\mathcal{W}$ is contained in the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{NN}^{\mathcal{H},\rho,\mathcal{L}}_{X,Y}$.

Finally, we apply the weighted vector-valued Stone-Weierstrass in Theorem 3.8 to show that $\mathcal{NN}^{\mathcal{H},\rho,\mathcal{L}}_{X,Y}$ is dense in $\mathcal{B}_{\psi}(X;Y)$. For this purpose, we first see that $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ vanishes nowhere as it contains the non-zero constant map $x\mapsto a(x):=\cos(0)=1\in\mathcal{A}$. Moreover, we observe that $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ is point separating and consists only of bounded maps, which shows that $\mathcal{A}\subseteq\mathcal{B}_{\psi}(X)$ is point separating of $\psi_{w}$-moderate growth, for all $w\in\mathcal{W}$ (see Remark 3.5). In addition, by using that $\mathcal{L}$ is dense in $Y$, it follows that $\mathcal{W}(x)=\left\{w(x):w\in\mathcal{W}\right\}=\mathcal{L}$ is dense in $Y$, for all $x\in X$. Hence, by applying the weighted vector-valued Stone-Weierstrass in Theorem 3.8, we conclude that $\mathcal{W}$ is dense in $\mathcal{B}_{\psi}(X;Y)$. However, since $\mathcal{W}$ is by the previous step contained in the $\|\cdot\|_{\mathcal{B}_{\psi}(X;Y)}$-closure of $\mathcal{NN}^{\mathcal{H},\rho,\mathcal{L}}_{X,Y}$, it follows that $\mathcal{NN}^{\mathcal{H},\rho,\mathcal{L}}_{X,Y}$ is also dense in $\mathcal{B}_{\psi}(X;Y)$. ∎

The global universal approximation result in Theorem 4.13 can be generalized to locally convex topological vector space as output spaces (see for a study of these infinite dimensional vector spaces and also Remark 2.6 and Remark 3.9).

3 Global universal approximation for non-anticipative functionals

The space of stopped paths was originally introduced in , and to generalize Föllmer’s pathwise Ito calculus to non-anticipative functionals. By introducing a special type of FNNs, so-called non-anticipative functional input neural networks, we now present the universal approximation result for this kind of functionals.

We consider again the space of stopped $\alpha$-Hölder continuous paths given by

Hence, we can apply the global universal approximation result below at least for continuous non-anticipative functionals, whose growth is controlled by $\psi:\Lambda^{\alpha}_{T}\rightarrow(0,\infty)$.

Motivated by Example 4.12 (ii), we introduce a specific type of functional input neural network defined on $\Lambda^{\alpha}_{T}$, which preserve the non-anticipative behaviour.

We now apply the global universal approximation result in Theorem 4.13 to show that every $f\in\mathcal{B}_{\psi}(\Lambda^{\alpha}_{T})$ can be approximated by some non-anticipative FNN.

Hence, the conclusion follows from Theorem 4.13. ∎

In Section 7, we provide some numerical examples, which illustrate how the theoretical approximation result in Corollary 4.17 can be applied to learn a non-anticipative functional by a non-anticipative FNNs. In particular for applications in finance, non-anticipative FNNs can be used within the framework of Stochastic Portfolio Theory (SPT) introduced by R. Fernholz in to learn optimal path-dependent portfolios, see .

Universal approximation on weighted spaces for linear functions of the signature

In this section, we give another application of the weighted real-valued Stone-Weierstrass Theorem 3.6, which is related to the theory of rough paths introduced by Terry Lyons in (we also refer to the textbooks and ). More precisely, we will show that a path space functional can be approximated globally by using linear functions of the paths’ signature, a notion which goes back to the work of Chen . This is in contrast to the usual results where only compact sets of paths are considered. Different versions of the latter, e.g. for finite variation paths or for continuous functions depending on the whole signature (instead of the rough path), are available in the literature (see for instance [69, Theorem 3.1], [62, Theorem 1], [72, Proposition 4.5] and [28, Section 3]). We refer to for the situation involving not just the approximation of a functional up to a fixed final value $T>0$, but a uniform approximation on the whole time interval $[0,T]$.

In general, one needs two key properties for universal approximation results for linear functions of the signature. First, that the signature at the terminal time determines uniquely the path (up to so-called tree-like equivalences, see and ), which ensures point separation. Second, that every polynomial of the signature can be realized as a linear function via the so-called shuffle product, which yields the algebra property of linear functions of the signature. While the classical Stone-Weierstrass then only guarantees the approximation on compact subsets of paths (e.g. with respect to a certain $p$-variation metric), there have already been some attempts to go beyond this rather restrictive setting. As already mentioned in the introduction, used the strict topology originating from to formulate a global approximation result, however not with the “true” signature but with a bounded normalization implying that the tractability properties of the signature (like computing expected signature analytically as e.g. done in for generic classes of stochastic processes) are lost.

In contrast, our weighted space setup allows to work with the “true” signature and thus to preserve its tractability properties, while we can still provide a global approximation result. Hereby, we consider the path space of $\alpha$-Hölder continuous paths as weighted space, which we either equip with an $0\leq\alpha^{\prime}<\alpha$-Hölder topology or the weak-$*$-topology as considered in Appendix A.

where $e_{I}\shuffle e_{J}:=\sum_{k=1}^{K}e_{I_{k}}$, with $K$ and $(I_{k})_{k=1,\ldots,K}$ being determined via $I\shuffle J=\sum_{k=1}^{K}I_{k}$.

2 Global universal approximation of α𝛼\alpha-Hölder rough paths

The following lemma relates $d_{cc,\infty}$ and $d_{cc,0}$ and can be deduced from the estimates in [47, Proposition 8.15].

To ensure point separation (without tree-like equivalences), we shall always add an additional component to $X$ representing time. More precisely, we define the subspace

To get a weighted space, we choose the weight function $\psi:\widehat{C}^{\alpha}_{d,T}\rightarrow(0,\infty)$ defined by

Hence, the weight function $\psi:\widehat{C}^{\alpha}_{d,T}\rightarrow(0,\infty)$ is on both spaces admissible and it follows from Remark A.7 (iv) that the weighted function space $\mathcal{B}_{\psi}(\widehat{C}^{\alpha}_{d,T})$ is the same for each underlying space $(\widehat{C}^{\alpha}_{d,T},\tau_{w^{*}})$ and $(\widehat{C}^{\alpha}_{d,T},d_{cc,\alpha^{\prime}})$, with $0\leq\alpha^{\prime}<\alpha$.

Now, we present the global universal approximation theorem for linear functions of signature, which can approximate any path space functional in $\mathcal{B}_{\psi}(\widehat{C}^{\alpha}_{d,T})$. Even though the proof looks involved, the most important ingredients can be summarized as follows: certain linear functions of the signature are given by the integrals

Since $\mathcal{B}_{\psi}(\widehat{C}^{\alpha}_{d,T})$ is the same for each underlying space $(\widehat{C}^{\alpha}_{d,T},\tau_{w^{*}})$ and $(\widehat{C}^{\alpha}_{d,T},d_{cc,\alpha^{\prime}})$, with $0\leq\alpha^{\prime}<\alpha$ (see Remark A.7 (iv)), and the metrics $d_{cc,0}$ and $d_{cc,\infty}$ are equivalent (see Lemma 5.3), we choose $(\widehat{C}^{\alpha}_{d,T},d_{cc,\infty})$ in the following. Then, the result follows from the weighted real-valued Stone-Weierstrass in Theorem 3.6 applied to the set

Therefore, we need to prove that $\mathcal{A}$ is a subspace of $\mathcal{B}_{\psi}(\widehat{C}^{\alpha}_{d,T})$, and a point separating subalgebra of $\psi$-moderate growth, which vanishes nowhere, where