ANALYTIC CONTINUATION IN REPRESENTATION THEORY AND HARMONIC ANALYSIS

by

Gestur Ólafsson

Abstract. — In this paper we discuss topics in harmonic analysis and representation theory related to two different real forms G/H and G^{c}/H of a complex semisimple symmetric space G_{ℂ}/H_{ℂ}. We connect representations of G and G^{c} using the theory of involutive representations of semigroups and reflection symmetry. We discuss how to generalize the Segal-Bargmann transform to real forms of bounded symmetric domains. This transform maps L^{2}(H/H ∩ K) into the representation space of a highest weight representation of G. We show how this transform is related to reflection symmetry, which shows that it is a natural transform related to representation theory. Finally we discuss the connection of the H-spherical characters of the representations and relate them to spherical functions.

1. Introduction

Let G be a connected semisimple Lie group with Lie algebra 𝔤. Let G_{ℂ} be the simply connected complex Lie group with Lie algebra 𝔤_{ℂ}. We will for simplicity assume that G ⊂ G_{ℂ} even if most of what we say holds also for the universal covering group G̃ and other connected groups locally isomorphic to G. Let θ : G → G a Cartan involution on G and denote by K = G^{θ} the corresponding maximal compact subgroup of G. We will be interested in a special class of symmetric spaces, that are closely related to real forms of bounded symmetric domains. We will therefore assume that D = G/K is a bounded symmetric domain. Let τ : D → D by a conjugation, i.e., an anti-holomorphic involution, fixing the point {K} ∈ D. Those involutions were classified by A. Jaffee in [25, 26]. We will give the list later. We lift τ to an involution on G, G_{ℂ}, 𝔤, and 𝔤_{ℂ}. We will also denote those involutions by τ. Then τ commutes with θ. Let H = G^{τ} ⊂ H_{ℂ} := G_{ℂ}^{τ}. Then D^{τ} = H/H ∩ K.

On the Lie algebra level we have

𝔤 = 𝔥 ⊕ 𝔮
= 𝔨 ⊕ 𝔭
= (𝔨 ∩ 𝔥) ⊕ (𝔨 ∩ 𝔮) ⊕ (𝔭 ∩ 𝔥) ⊕ (𝔭 ∩ 𝔮) ,

𝔨 = {X ∈ 𝔤 | θX = X}, 𝔭 = {X ∈ 𝔤 | θX = −X}, 𝔥 = {X ∈ 𝔤 | τX = X}, and 𝔮 = {X ∈ 𝔤 | τX = −X}. Define a new real form of 𝔤_{ℂ} by

𝔤^{c} := 𝔥 ⊕ i𝔮

and let G^{c} be the corresponding real analytic subgroup of G_{ℂ}. Denote also by τ the restriction of τ to G^{c}. Let H^{c} := G^{cτ}. Then H = H^{c} = G ∩ G^{c}.

We have the following diagrams

M_{ℂ} :=_{Complex} G_{ℂ}/H_{ℂ}

↗  ↖
M := G/H  Real forms  M^{c} := G^{c}/H

and

D^{τ} = H/H ∩ K ↪_{Real form} D = G/K .

The ideas that we discuss here are how to analyze representations of G, G^{c}, and H via analytic continuation to open domains in M_{ℂ} or by restriction to a real form. The main tools are involutive representations and positive definite kernels. This can be expressed by the following simple diagram:

Unitary highest weight representations of G

⟷

Generalized principal series representations of G^{c}

↕

Reflection positivity H-spherical characters and spherical functions

↕

Restriction of holomorphic functions The Segal-Barmann transform

The subspace of functions with support in the open orbit HAN

↘  ↙

Representations of H, Canonical representations

Most of the ideas discussed here have been explained before in [30, 31, 49, 51]. We would in particular like to point to [31] for discussion on reflection positivity and highest weight representations. Several other people have been working on similar projects. We would like to point out here the following papers and preprints [1, 2, 7, 8, 19, 41, 43, 56, 64, 68].

This paper is based on lectures at the conferences Analyse harmonique et analyse sur les varietes, 05/31–06/05, 1999 at Luminy, at the The 1999 Twente Conference on Lie Groups, Dec. 20–22, 1999, and a series of lectures at the University in Bochum, June 2000. We would like to thank the University of Nancy, the organizers of those conferences and the Forschungsschwerpunkt “Globale Methoden in der komplexen Geometrie” in Bochum for their support. We would also like to thank LEQSF for travel support and the Division of Mathematics at the Research Institute of Iceland for financial support during November/December 1999 as part of the work on this paper was done. We would like to thank M. Davidson, R. Fabec, P. Jorgensen and B. Ørsted for helpful discussion the last few years on several topics, results and ideas discussed in this paper. Finally we would like to thank A. Pasquale for reading over the paper and for her valuable comments.

2. Unitary highest weight representations

We use the same notation and assumptions as in the introduction. In particular assume that G ⊂ G_{ℂ} is a Hermitian group, and G_{ℂ} is simply connected. Thus D = G/K is a bounded symmetric domain. The complex structure on D corresponds to an element Z^{0} ∈ 𝔷(𝔨) such that ad(Z^{0}) has eigenvalues 0, i, −i. The eigenspace corresponding to 0 is 𝔨_{ℂ}, and we denote by 𝔭^{+}, respectively 𝔭^{−} the eigenspace corresponding to i respectively −i. Then both 𝔭^{+} and 𝔭^{−} are abelian subalgebras of 𝔤_{ℂ} and

𝔤_{ℂ} = 𝔭^{+} ⊕ 𝔨_{ℂ} ⊕ 𝔭^{−} .

Let K_{ℂ} := exp(𝔨_{ℂ}), and P^{±} := exp(𝔭^{±}). The restriction of the exponential map is an isomorphism of 𝔭^{±} onto P^{±}. The set P^{+}K_{ℂ}P^{−} is open and dense in G_{ℂ}. Furthermore multiplication induces a diffeomorphism

P^{+} × K_{ℂ} × P^{−} ∋ (p, k, q) ↦ pkq ∈ P^{+}K_{ℂ}P^{−} ⊂ G_{ℂ} .

We denote the inverse map by x ↦ (p(x), k(x), q(x)). The Harish-Chandra bounded realization of D is given by

(2.1)  G/K ∋ gK ↦ log(p(g)) ∈ Ω_{ℂ} ⊂ 𝔭^{+}

and Ω_{ℂ} is a bounded symmetric domain in 𝔭^{+}. Let (π, H) be a representation of G in a Hausdorff, locally convex complete topological vector space H, and let L be a closed subgroup of G. A vector v ∈ H is called L-finite if π(L)v spans a finite dimensional subspace of H. We call v smooth or analytic if for all X ∈ 𝔤 the map

ℝ ∋ t ↦ π(exp(tX))v ∈ H

is smooth, or analytic respectively. We denote by H_{L}, H^{∞}, H^{ω} the space of L-finite, smooth, respectively analytic vectors in H. Define a representation of 𝔤 on H^{∞} by

dπ(X)u := lim_{t→0} \frac{π(exp tX)u − u}{t} , u ∈ H^{∞} .

We extend dπ by linearity to a representation of 𝔤_{ℂ} and then of U(𝔤), the universal enveloping algebra of 𝔤. The representations of G that we are mainly interested in are the unitary highest weight representations of G (see [5, 6, 9, 10, 15, 16, 22, 27, 42, 61, 65, 67] for further information.) Let (π, H) be an admissible representation of G in a Banach space, and assume that the center of U(𝔤) acts by scalars. Then H_{K} ⊂ H^{ω} and H_{K} is an (U(𝔤), K)-module in the sense that it is both an U(𝔤) and a K-module such that

X · (k · u) = (Ad(k)X) · u, ∀k ∈ K, X ∈ 𝔤, u ∈ H_{K} .

We say that an (U(𝔤), K)-module (π, H) is admissible if the multiplicity of each irreducible representation of K in H is finite. Let 𝔱 ⊂ 𝔨 be a Cartan subalgebra of 𝔤 containing Z^{0}. Then 𝔱 ⊂ 𝔷_{𝔤}(Z^{0}) ⊂ 𝔨 so 𝔱 is a Cartan subalgebra of 𝔨.

Definition 2.1. — Let H be an (U(𝔤), K)-module. Then π is called a highest weight representation if there exists a Borel subalgebra 𝔭 = 𝔱_{ℂ} ⊕ 𝔲 ⊂ 𝔤_{ℂ}, λ ∈ 𝔱_{ℂ}^{∗}, and v ∈ H such that the following holds:

1. X · v = λ(X)v for all X ∈ 𝔱;

2. π(𝔲)v = 0;

3. U(𝔤)v = H.

The element v is called primitive element of weight λ.

All the irreducible unitary highest weight representations of G can be constructed in a space of holomorphic functions on D for an appropriate choice of Z^{0} or, which is the same, complex structure on D. For that let π be an irreducible representation of K. Let Δ denote the set of roots of 𝔱_{ℂ} in 𝔤_{ℂ}. A root α is called compact if 𝔤_{ℂα} ⊂ 𝔨_{ℂ}. Otherwise α is called non-compact. Let Δ_{c} respectively Δ_{n} be the set of compact respectively non-compact roots. We choose the set Δ^{+} of positive roots such that Δ_{c}^{+} := Δ^{+} ∩ Δ_{c} is a system of positive roots for Δ_{c} and Δ_{n}^{+} := Δ^{+} ∩ Δ_{n} = {α ∈ Δ | 𝔤_{ℂα} ⊂ 𝔭^{+}}. Choose H_{α} ∈ i𝔱, such that H_{α} ∈ [𝔤_{α}, 𝔤_{−α}] and α(H_{α}) = 2. Let W_{K} = W (Δ_{c}) ⊂ W = W (Δ) be the Weyl group generated by the reflections s_{α}(X) = X − α(X)H_{α} for α ∈ Δ_{c} respectively α ∈ Δ. We denote the corresponding reflection on i𝔱^{∗} by the same letter, i.e, s_{α}(β) = β − \frac{2(β,α)}{(α,α)}α. Then Δ_{n}^{+} is invariant under W_{K}. Let σ : G_{ℂ} → G_{ℂ} be the conjugation with respect to G. We sometimes write W or ḡ for σ(W), respectively σ(g). We will usually use capital letters for the elements of the Lie algebra 𝔤 or 𝔤_{ℂ} except where we are viewing them as complex variables or elements in Ω_{ℂ}.

Let (π, V) be an irreducible representation of K with highest weight μ = μ(π) ∈ i𝔱^{∗}. For z, v, w ∈ 𝔭^{+} and g ∈ G_{ℂ} such that g exp(z), exp(−w) exp(v) ∈ P^{+}K_{ℂ}P^{−}, let

(2.2)  g · z :=  log(p(g exp z))
(2.3)  J(g, z) :=  k(g exp z) , and
(2.4)  κ(v, w) :=  k(exp(−\overline{w}) exp(v)).

Then the isomorphism in (2.1) intertwines the natural G-action on D with the action (g, z) ↦ g · z of G on Ω_{ℂ}. The function J(g, z) is called the universal factor of automorphy. Finally we define

(2.5)  J_{π}(g, z) := π(J(g, z)) and K_{π}(z, w) := π(κ(z, w))^{−1} .

Then z, w ↦ K_{π}(z, w) is holomorphic in the first variable and anti-holomorphic in the second variable.

Let S(Ω_{ℂ}) := {g ∈ G_{ℂ} | g^{−1} · Ω_{ℂ} ⊂ Ω_{ℂ}}. Then S(Ω_{ℂ}) is a maximal closed semigroup in G_{ℂ} and there exists a maximal closed and convex G-invariant cone W_{max} ⊂ 𝔤 such that W_{max} ∩ −W_{max} = {0} (pointed), W_{max} − W_{max} = 𝔤 (generating), and (see [21, 23, 44] )

S(Ω_{ℂ}) = G exp(iW_{max}) .

Let W_{min} = {X ∈ 𝔤 | ∀Y ∈ W_{max} : −B(X, θ(Y)) ≥ 0}, where B stands for the Killing form. Then W_{min} is a minimal G-invariant pointed and generating cone in 𝔤. Define g^{∗} = σ(g)^{−1}. If s = g exp(iW) ∈ S(Ω_{ℂ}) then

s^{∗} = exp(iW)g^{−1} = g^{−1} exp(iAd(g)W) ∈ S(Ω_{ℂ}) .

We notice the following well known and important lemma.

Lemma 2.2. — Let s, s_{1}, s_{2} ∈ S(Ω_{ℂ}) and let w, v ∈ Ω_{ℂ}. Then the following holds: 1. (s_{1}s_{2}) · v = s_{1} · (s_{2} · v);

2. J(s_{1}s_{2}, v) = J(s_{1}, s_{2} · v)J(s_{2}, v);

3. J_{π}(s_{1}s_{2}, v) = J_{π}(s_{1}, s_{2} · v)J_{π}(s_{2}, v);

4. \overline{J(s^{∗}, w)}κ(s · v, w)J(s, v) = κ(v, s^{∗} · w);

5. J_{π}(s, v)^{−1}K_{π}(s · v, w) = K_{π}(v, s^{∗} · w) (J_{π}(s^{∗}, w)^{∗})^{−1}.

Proof. — 1) and 2) We have, with q_{1}, q_{2} and q_{3}, denoting elements in P^{−},

s_{1}s_{2} exp(v) = s_{1} exp(s_{2} · v)k(s_{2} exp v)q_{1}
= exp(s_{1} · (s_{2} · v))k(s_{1} exp(s_{2} · v))q_{2}k(s_{2} exp v)q_{1}
= exp(s_{1} · (s_{2} · v))k(s_{1} exp(s_{2} · v))k(s_{2} exp v)q_{3}

where we have used that K_{ℂ} normalizes P^{±}.

3) Follows from (2).

4) By the defining relation

s^{∗} exp(w) = exp(s^{∗} · w)J(s^{∗}, w)q_{1}

we get exp(−\overline{s^{∗} · w}) = \overline{q_{2}}\overline{J(s^{∗}, w}) exp(−w)s. Hence

exp(−\overline{s^{∗} · w}) exp(v) = \overline{q_{2}}\overline{J(s^{∗}, w}) exp(−w)s exp(v)
= \overline{q_{2}}\overline{J(s^{∗}, w}) exp(−w) exp(s · v)J(s, v)q_{3} .

Hence the claim follows.

5) This follows from (4) using that π(g^{∗}) = π(g)^{∗}.

We notice that all of those relations can be lifted to G̃. We will use this fact without further comments. Let (π, V) be an irreducible representation of K̃. Let O (Ω_{ℂ}, V) be the space of V-valued holomorphic functions on Ω_{ℂ}. Define a representation ρ_{π} of G̃ on O(Ω_{ℂ}, V) by

ρ_{π}(g)f(z) := J_{π}(g^{−1}, z)^{−1}f(g^{−1} · z) .

Then ρ_{π} is a representation of G̃ by Lemma 2.2, part 3. Let (ρ, H) be an irreducible unitary highest weight module with lowest weight π, then we can choose 𝔭^{+} such that {0} ≠ H^{𝔭^{+}} ≃_{K} V. Furthermore there is an injective map H ↪ O(Ω_{ℂ}, H^{𝔭^{+}}) intertwining ρ and ρ_{π}, see [5]. Here are the main ideas in the proof. Let σ : 𝔤_{ℂ} → 𝔤_{ℂ} be the conjugation with respect to 𝔤. For w ∈ 𝔭^{+} define q_{w} : H^{𝔭^{+}} → H by

q_{w}(v) := ∑_{n=0}^{∞} \frac{ρ_{π}(σ(w))^{n}v}{n!} .

If v ≠ 0 the series defining q_{w}(v) converges if and only if w ∈ Ω_{ℂ}. Define U : H → O(Ω_{ℂ}, H^{𝔭^{+}}) by

(U(w))(z) := q_{z}^{∗}(w) .

Then U intertwines the representation ρ and ρ_{π} and gives the geometric realization of ρ. One should remark, that even if the classification of unitary highest weight modules is known, it is still an open problem to describe analytically the unitary structure for ρ_{π} in general.

Let λ_{0} ∈ i𝔷(𝔨)^{∗} such that λ_{0}(iZ^{0}) = 1. Let μ(π) be the highest weight of π. Then μ(π) = μ_{0} + rλ_{0} where μ_{0} ∈ i𝔱 ∩ 𝔨′)^{∗} is Δ_{c}^{+} dominant. Consider the affine line L(μ_{0}) = {μ_{0} + tλ_{0}|t ∈ ℝ}. The following is due to Jakobsen, Enright, Howe and Wallach:

Theorem 2.3. — Let the notation be as above. Then there exists constants a(μ_{0}) ≤ 0 and c(μ_{0}) > 0 such that the Representation ρ_{π} is unitary if and only if

μ(π) ∈ {μ_{0} + rλ_{0} | r < a(μ_{0})} ˙∪ {μ_{0} + z_{j}λ_{0} | j = 0, …, n}

with z_{0} = a(μ_{o}) and z_{j+1} − z_{j} = c(μ_{0}).

Write U(μ_{0}) = {μ_{0} + rλ_{0} | r < a(μ_{0})} = μ_{0} + (−∞, a(μ_{0}))λ_{0}.

Theorem 2.4. — Assume that (ρ_{π}, H(ρ_{π})) is unitary. Then all the polynomials on 𝔭^{+} are in H(ρ_{π}) if and only if r < a(μ_{0}). In that case the space H(ρ_{π})_{K} ≃ U(𝔭^{−})×_{K}V is exactly the space of polynomial functions on 𝔭^{+}.

If μ = tλ_{0}, i.e., π is a character, then K_{π}(·, w) ∈ H(ρ_{π})^{ω} and

(f, K_{w}) = f(w)

for all f ∈ H(ρ_{π}).

Theorem 2.5 (Harish-Chandra). — Let ρ(Δ^{+}) = 1/2 ∑_{α∈Δ^{+}} α. The representation (ρ_{π}, H(ρ_{π})) is isomorphic to a direct summand in L^{2}(G) if and only ρ_{π} defines a representation of G and

< μ(π) + ρ, α >< 0 ∀α ∈ Δ_{n}^{+} .

The highest weight representations in L^{2}(G) are called the holomorphic discrete series of G. In this case the inner product on H(π) is given by

(f, g) = c ∫_{Ω_{ℂ}}(f(z), K(z, z)^{−1}g(z)) dμ(z)

where μ is a G-invariant measure on Ω_{ℂ}.

We also have the following, see [24, 51, 52]:

Theorem 2.6. — Let 𝔞_{q} = 𝔱 ∩ 𝔮 and assume that 𝔞_{q} is maximal abelian in 𝔮 ∩ 𝔨. Let Δ(𝔭^{+}, 𝔞_{q}) = {α|𝔞_{q} | α ∈ Δ_{n}^{+}}. Then (ρ_{π}, H(ρ_{π})) is isomorphic to a direct summand in L^{2}(G/H) if and only if ρ_{π} defines a representation of G, V^{K∩H} (and thus μ(π) ∈ i𝔞_{q}^{∗}), and

< μ(π) + ρ|𝔞_{q}, α >< 0 , ∀α ∈ Δ(𝔭^{+}, 𝔞_{q}) .

The highest weight representations in L^{2}(G/H) are called the holomorphic discrete series of G/H.

We notice that there are holomorphic discrete series of G/H that are not in the holomorphic discrete series of G. This is due to the so-called ρ_{ℓ}-shift, see [51] for further discussion and examples.

The highest weight representations are closely related to holomorphic representations of semigroups of the form G exp(iW), see [22, 23, 24, 42, 57]. Let π be a unitary representation of G in a Hilbert space H(π). Let

W (π) := {X ∈ 𝔤 | ∀u ∈ H(π)^{∞} : (idπ(X)u, u) ≤ 0} .

Then W (π) is a G-invariant closed convex cone in 𝔤. If W (π) is pointed and generating, then G exp(iW (π)) = S(W (π)) is a closed semigroup with interior S(W (π))^{o} = G exp(iW (π)^{o}) = S(W (π)^{o}).

Theorem 2.7. — Let ρ_{π} be a unitary highest weight representation. Then W (π) is pointed and generating and ρ_{π} extends to an involutive, holomorphic, and contractive representation of the semigroup S(W (π)) := G exp(iW (π)). In particular we have for all s ∈ S(W (π)):

1. ρ_{π}(s)^{∗} = ρ_{π}(s^{∗}).

2. ||ρ_{π}(s)|| ≤ 1 for all s ∈ S(W (π)).

We also notice the following due to K-H. Neeb and Olshanskii, see [42]:

Theorem 2.8. — Let π be a unitary representation of G. If W (π) is pointed and generating, then π is a direct integral of highest weight representations.

We will in the following mainly work with characters, i.e., d(π) = dim(H(π)) = 1. Each character corresponds to a element μ ∈ i𝔷_{𝔨} such that

χ_{μ}(exp Z) = e^{μ(Z)} .

If we replace G with the universal covering group G̃ ≃ K̃ ×𝔭, where K̃ is the universal covering group of K, then each μ ∈ i𝔷_{𝔨} gives rise to a character of K̃ and its complexification K̃_{ℂ}. For K itself we need to assume that μ(Z) ∈ 2πiℤ for Z ∈ exp^{−1} {e}. If π = χ_{μ} is a character, then we write J_{μ}, K_{μ}, etc.

3. The Restriction Principle and the Segal-Bargmann Transform

Let us start by recalling the restriction principle put forward in [54]. Let M_{ℂ} be a connected complex manifold and let M ⊂ M_{ℂ} be a totally real submanifold thus locally the inclusion M ↪ M_{ℂ} is the same as ℝ^{n} ↪ ℂ^{n} and if F is a holomorphic function on M_{ℂ} such that the restriction F |M = 0 then it follows that F = 0. Let F be a Hilbert space of holomorphic functions F : M_{ℂ} → ℂ. (We can also consider vector valued functions or even sections of a holomorphic vector bundle over M_{ℂ}.) We assume that F is a reproducing Hilbert space, that is the evaluation maps F ↦ F (w) are all continuous on F. This implies the existence of a K_{w} ∈ F such that

F (w) = (F, K_{w})

for all F ∈ F and all w ∈ M_{ℂ}. We notice that K(z, w) := K_{w}(z) is holomorphic in z and anti-holomorphic in w. The function K(z, w) is the reproducing kernel for F. We have

1. K(z, w) = (K_{w}, K_{z}) = \overline{(K_{z}, K_{w})} = \overline{K(w, z)} .

2. ||K_{w}||^{2} = K(w, w).

Let μ be a measure on M and let D : M → ℂ^{∗}. Assume that m ↦ D(m)F (m) is in L^{2}(M, μ) for all F ∈ F. Define the restriction map R : F → L^{2}(M, μ) by

[RF] (m) := D(m)F (m) .

Then R is injective. If G is a Lie group and H is a closed subgroup such that G acts on M_{ℂ}, H acts on M, and F is a unitary G-module, then it is natural to assume that μ is H invariant, so that L^{2}(M, μ) is a unitary H-module. We would then determine D such that R is an H-morphism. Assume that Im (R) is dense. Then R^{∗} : L^{2}(M, μ) → F is injective and

R^{∗}h(z) = (R^{∗}h, K_{z})
= (h, RK_{z})
= ∫ h(m)\overline{D(m)}K(z, m) dμ .

Thus we have

(3.1)  RR^{∗}h(x) = ∫ h(m)D(x)\overline{D(m)}K(x, m) dμ .

Now polarize R^{∗} to get R^{∗} = U |R^{∗}|. Then U : L^{2}(M, μ) → F is a unitary isomorphism.

Definition 3.1. — The map U : L^{2}(M, μ) → F is called the generalized Segal-Bargmann transform.

We have U^{∗}R^{∗} = |R^{∗}| and hence RU = |R^{∗}|. Thus for x ∈ M and h ∈ L^{2}(M, μ) we get

Uh(x) = D(x)^{−1} |R^{∗}| h(x) .

To find an explicit expression for the Segal-Bargmann transform we need to take the square root of RR^{∗} in (3.1). This can be done in some special cases. Let us start with the classical Segal-Bargmann transform. Let M_{ℂ} = ℂ^{n} and M = ℝ^{n}. Let F be the classical Fock-space of holomorphic functions F : ℂ^{n} → ℂ such that

||F ||^{2} := π^{−n} ∫ |F (z)|^{2} e^{−|z|^{2}} dxdy < ∞ .

Then F is a reproducing Hilbert space with inner product

(F, G) = π^{−n} ∫ F (z)\overline{G(z)} e^{−|z|^{2}} dxdy

and reproducing kernel K(z, w) = e^{z·w} where z · w = z_{1}w̄_{1} + ⋯ + z_{n}w̄_{n}. Let D(x) := e^{−|x|^{2}/2}. Then the restriction map R becomes

RF (x) = e^{−|x|^{2}/2}F (x) .

The holomorphic polynomials P (z) = ∑ a_{α}z^{α} are dense in F and obviously RP ∈ L^{2}(ℝ^{n}). Hence all the Hermite functions h_{α}(x) = (−1)^{|α|} (D^{α}e^{−|x|^{2}}) e^{|x|^{2}/2} are in the image of R so Im(R) is dense. For z, w ∈ ℂ^{n} let (z, w) = ∑ z_{j}w_{j}. We then have

R^{∗}g(z) = ∫ g(y)e^{−|y|^{2}/2}e^{z·y} dy
= e^{(z,z)/2} ∫ g(y)e^{−(z−y,z−y)/2} dy
= e^{(z,z)/2}g ∗ p(z)

where p(z) = e^{−(z,z)/2} is holomorphic. It follows that

(3.2)  RR^{∗}g(x) = g ∗ p(x) .

As p ∈ L^{2}(ℝ) it follows in particular that ||RR^{∗}|| ≤ ||p||_{2}, so RR^{∗} is continuous, and

(R^{∗}g, R^{∗}g) = (RR^{∗}g, g) ≤ ||RR^{∗}|| ||g||_{2}^{2} .

Thus we have the lemma:

Lemma 3.2. — R^{∗} is continuous.

Let p_{t}(x) = (2πt)^{−n/2}e^{−(x,x)/2t} be the heat kernel on ℝ^{n}. Then (p_{t})_{t>0} is a convolution semigroup and p = (2π)^{n/2}p_{1}. Hence \sqrt{RR^{∗}} = (2π)^{n/4}p_{1/2}∗ or

RUg(x) = |R^{∗}| g(x) = (2π)^{n/4}p_{1/2} ∗ g(x) = 2^{n/4}π^{−n/4} ∫ g(y)e^{−(x−y,x−y)} dy .

As RUg(x) = e^{−(x,x)/2}U(g)(x) it follows that

Ug(x) = (2/π)^{n/4}e^{(x,x)/2} ∫ g(y)e^{−(x−y,x−y)} dy

for x ∈ ℝ^{n}. But the function on the right hand side is holomorphic in x. By analytic continuation we get the following theorem.

Theorem 3.3. — The map U : L^{2}(ℝ^{n}) → F given by

Ug(z) = (2/π)^{n/4} ∫ g(y) exp(− (y, y) + 2(z, y) − (z, z) /2) dy

is a unitary isomorphism.

Next we consider the case where M_{ℂ} = Ω_{ℂ} is a bounded symmetric domain of the form G/K and the corresponding real forms H/H ∩ K (see [23, 25, 26, 34, 46, 47] for classifications, structure theory and further information). Let η : Ω_{ℂ} → Ω_{ℂ} be an anti-holomorphic involution fixing the point 0 = {K} ∈ Ω_{ℂ}. Then Ω = Ω_{ℂ}^{η} is a real form of Ω_{ℂ}. The group G is locally isomorphic to the group I(Ω_{ℂ})_{o} of holomorphic isomorphism of Ω_{ℂ}. Define τ : I(Ω_{ℂ})_{o} → I(Ω_{ℂ})_{o} by τ(f)(z) = η(f(η(z)). Then τ is an involution commuting with the Cartan involution θ with K = G^{θ}. Lift τ to the Lie algebra of I(Ω_{ℂ})_{o} which is isomorphic to 𝔤 and then extend that involution – also denoted by τ – to an involution on 𝔤_{ℂ}. As G_{ℂ} is simply connected it follows that τ defines an involution on G_{ℂ} leaving G invariant. Let H = G^{τ} ⊂ H_{ℂ} = G_{ℂ}^{τ} and notice that H_{ℂ} is connected as G_{ℂ} is simply connected. We have that Ω = H/H ∩ K = H_{o}/H_{o} ∩ K. The classification of these spaces is given by the following table:

𝔤^{c} with complex structure

𝔤^{c} = 𝔥_{ℂ}  𝔤 = 𝔥 × 𝔥  𝔥
Dual  Hermitian
𝔰𝔩(p + q, ℂ) 𝔰𝔲(p, q) × 𝔰𝔲(p, q) 𝔰𝔲(p, q)
𝔰𝔬(2n, ℂ)  𝔰𝔬^{∗}(2n) × 𝔰𝔬^{∗}(2n) 𝔰𝔬^{∗}(2n)
𝔰𝔬(n + 2, ℂ) 𝔰𝔬(2, n) × 𝔰𝔬(2, n) 𝔰𝔬(2, n)
𝔰𝔭(n, ℂ)  𝔰𝔭(n, ℝ) × 𝔰𝔭(n, ℝ) 𝔰𝔭(n, ℝ)
𝔢_{6}  𝔢_{6(−14)} × 𝔢_{6(−14)}  𝔢_{6(−14)}
𝔢_{7}  𝔢_{7(−25)} × 𝔢_{7(−25)}  𝔢_{7(−25)}

𝔤^{c} without complex structure

𝔤^{c}  𝔤  𝔥
Dual  Hermitian
𝔰𝔩(p + q, ℝ)  𝔰𝔲(p, q)  𝔰𝔬(p, q)
𝔰𝔲(n, n)  𝔰𝔲(n, n)  𝔰𝔩(n, ℂ) × ℝ
𝔰𝔲^{∗}(2(p + q)) 𝔰𝔲(2p, 2q) 𝔰𝔭(p, q)
𝔰𝔬(n, n)  𝔰𝔬^{∗}(2n)  𝔰𝔬(n, ℂ)
𝔰𝔬^{∗}(4n)  𝔰𝔬^{∗}(4n)  𝔰𝔲^{∗}(2n) × ℝ
𝔰𝔬(p + 1, q + 1) 𝔰𝔬(2, p + q) 𝔰𝔬(p, 1) × 𝔰𝔬(1, q)
𝔰𝔭(n, ℝ) 𝔰𝔭(n, ℝ) 𝔰𝔩(n, ℝ) × ℝ
𝔰𝔭(n, n)  𝔰𝔭(2n, ℝ) 𝔰𝔭(n, ℂ)
𝔢_{6(6)}  𝔢_{6(−14)}  𝔰𝔭(2, 2)
𝔢_{6(−26)}  𝔢_{6(−14)}  𝔣_{4(−20)}
𝔢_{7(−25)}  𝔢_{7(−25)}  𝔢_{6(−26)} × ℝ
𝔢_{7(7)}  𝔢_{7(−25)}  𝔰𝔲^{∗}(8)

We notice that in the first table the symmetric space Ω is a complex bounded symmetric domain, Ω_{ℂ} = Ω×\overline{Ω}, where stands for “opposite complex structure”, and the imbedding of Ω into Ω_{ℂ} is the diagonal imbedding ω ↦ (ω, \overline{ω}). The holomorphic functions on Ω_{ℂ} are then the functions f(z, w) which are holomorphic in the first variable and anti-holomorphic in the second variable. Some of those cases were treated in detail in [56]. We note that all the classical irreducible Riemannian symmetric spaces (with a possible extension by ℝ) show up in the third column in this list. The spaces that are missing are: E_{6(2)}/SU(6) × SU(2), E_{6(6)}/Sp(4), E_{7(7)}/SU(8), E_{7(−5)}/SO(12) × SU(2), E_{8(8)}/SO(16), E_{8(−24)}/E_{7} × SU(2), F_{4(4)}/Sp(3) × SU(2), G_{2(2)}/SU(2) × SU(2). It is interesting that this list contains all the quaternions exceptional Riemannian symmetric spaces.

Our aim is to use the generalized Bargmann transform to analyze the representation ρ_{π}|H_{o} where ρ_{π} is a unitary highest weight representation with minimal K-type π acting on the finite dimensional Hilbert space V(π). In this case we let

D_{π}(h) = J_{π}(h, 0)^{−1} = π(k(h))^{−1}

for all h ∈ P^{+}K_{ℂ}P^{−}. Then D_{π}(gk) = π(k)^{−1}D_{π}(g). Assume for the moment that ||D_{π}|| ∈ L^{2}(Ω, dm), where dm is the H-invariant measure on Ω given by

∫_{Ω} f(x)dm(x) = ∫_{H} f(h · 0) dh .

This can be made precise using the root structure of H/H ∩ K and G/K . Define

RF (h) = D_{π}(h)F (h · 0) , F ∈ H(ρ_{π}) .

Let V(π) → Ω be the vector bundle

H ×_{π|H∩K} V(π) → Ω .

Then RF ∈ L^{2}(V(π)) for all F ∈ H(ρ_{π}) .

Lemma 3.4. — Assume that D_{π} ∈ L^{2}(Ω) and that π ∈ U(μ_{0}). Then the restriction map R : H(ρ_{π}) → L^{2}(Ω, dm) is injective and intertwines ρ_{π}|H and the left regular action λ of H on L^{2}(V(π)). Furthermore Im(R) is dense in L^{2}(V(π)).

Proof. — Let h ∈ H and F ∈ H(ρ_{π}). Let a ∈ H. Then

R(ρ_{π}(h)F)(a) = D_{π}(a)[ρ_{π}(h)F](a)
= D_{π}(a)J_{π}(h^{−1}, a · 0)^{−1}F (h^{−1}a · 0)
= J_{π}(a, 0)^{−1}J_{π}(h^{−1}, a · 0)^{−1}F (h^{−1}a · 0)
= J_{π}(h^{−1}a, 0)^{−1}F (h^{−1}a · 0)
= D_{π}(h^{−1}a)F (h^{−1}a · 0)
= λ(h)RF (a) .

Assume that RF = 0. Then D_{π}(a)F (a · 0) = 0 for all a ∈ H. As D_{π}(a) is regular it follows that F (a · 0) = 0. But then F |Ω = 0 and hence F = 0.

Let f ∈ L^{2}(V(π)) and ϵ > 0. Let g be a compactly supported section such that ||f − g||_{2} < ϵ/2. Then the function h ↦ D_{π}(h)^{−1}g(h) is K ∩ H invariant and can therefore be viewed as a V(π)-valued function on the compact set cl(Ω), the closure of Ω. Let p : Ω → V(π) be a polynomial such that

sup_{z∈Ω̄} ||p(x) − D_{π}(x)^{−1}g(x)|| < \frac{\sqrt{ϵ}}{2 ||D_{π}||_{2}} .

Then

||f − Rp|| ≤ ||f − g|| + ||g − D_{π}p||

But

||g − D_{π}p||^{2} = ∫_{H} ||g(h) − D_{π}(h)p(h · 0)||^{2} dh
= ∫ ||D_{π}(h)||^{2} ||D_{π}(h)^{−1}g(h) − p(h · 0)||^{2} dh
≤ ϵ^{2}/4 .

Hence ||f − Rp||_{2} < ϵ.

Polarizing R^{∗} gives us now a unitary H-isomorphism U : L^{2}(V(π)) → H(ρ_{π}). Hence

Theorem 3.5. — Assume that D_{π} ∈ L^{2} (V(π)) and that μ ∈ U(μ_{0}). Then ρ_{π}|H is unitary equivalent to the representation of H on L^{2}(V(π)).

Let us remark here, that in the case where Ω_{ℂ} = Ω×Ω̄, i.e., G = H ×H, then each of the highest weight representations ρ_{π} is of the form ρ_{H} ⊗ ρ̄_{H} where ρ_{H} is a highest weight representation of H and ρ̄_{H} is the conjugate (or dual) representation. The above result therefore tells us, that ρ_{H} ⊗ ρ̄_{H} ≃ L^{2}(V(π_{H})) where π_{H} is the minimal K ∩ H-type of ρ_{H}.

We will now analyze the generalized Segal-Bargmann transform in more detail for the scalar case. In that case χ_{μ}|H_{o} ∩ K is always trivial so D_{π}, which we will now denote by D_{μ}, is a function on Ω given by D_{μ}(h · 0) = D_{μ}(h), h ∈ H_{o}. The map R^{∗} is given by

R^{∗}f(z) = (f, RK_{z})
= ∫_{Ω} f(y)\overline{D_{μ}(y)}K(z, y) dm(y)
(3.3)  = ∫_{H_{o}} f(h · 0)J_{μ}(h^{−1}, z)^{−1} dh

where the last equation follows from Lemma 2.2. Hence for x = t · 0, t ∈ H_{o}, we get

RR^{∗}f(x) = ∫_{Ω} f(y)D_{μ}(x)\overline{D_{μ}(y)}K(x, y) dm(y)
= ∫_{H} f(h)J_{μ}(t, 0)^{−1}J_{μ}(h^{−1}, t · 0)^{−1} dh
= ∫ f(h) J_{μ}(h^{−1}t, 0)^{−1} dh
= f ∗ D_{μ}(t) .

Thus again the result is a convolution operator with an L^{2}-function and the generalized Segal-Bargmann transform is again given by a convolution with a function φ. Contrary to the classical case, this convolution operator does not result from a convolution semigroup. Hence the task of determining the function φ becomes much more involved. But since D_{μ} is a H_{o} ∩ K-biinvariant function it is determined by its restriction to a maximal vector subgroup of H and can be determined by the spherical Fourier transform on H_{o}/H_{o} ∩ K. Hence one can in ”principle” find a H_{o} ∩ Kbiinvariant function φ on H_{o} such that the generalized Segal-Bargmann transform is given by:

Theorem 3.6. — The Segal-Bargmann transform is given by

Uf(z) = D_{μ}(z)^{−1}f ∗ φ(z) .

Here are few problems that are still unsolved or have only case by case solutions:

1. Decompose the restriction of ρ_{μ} for more singular parameters.

2. Find an explicit formula for φ.

3. We have a canonical orthonormal basis {p_{I}}_{I} for H(ρ_{μ}) given by polynomials. Is there an expression for the corresponding orthonormal bases {U^{∗}p_{I}} for L^{2} (Ω, μ)?

4. Work out the case where π is not assumed to be one-dimensional.

4. The Principal Series of G^{c} and Reflection Positivity

The material in this section follows [30, 31] with few motifications. We use [23] as standard reference for the structure theory of G^{c}/H. Let τ : 𝔤 → 𝔤 be the involution from the previous section. Then 𝔤 = 𝔥 ⊕ 𝔮 and we define 𝔤^{c} := 𝔥 ⊕ i𝔮. Let G^{c} be the analytic subgroup of G_{ℂ} corresponding to 𝔤^{c} as before.

Lemma 4.1. — We have H = G ∩ G^{c}.

Proof. — Let σ be the conjugation with respect to G and let σ^{c} be the conjugation with respect to G^{c}. Then σ|G^{c} = τ|G^{c} and σ^{c}|G = τ|G. The claim follows directly from this.

Let P_{max} := LAN^{−} := K_{ℂ}P^{−} ∩ G^{c}. Then P_{max} is a maximal parabolic subalgebra of G^{c}. Notice that LA = K_{ℂ}∩G^{c}, A = exp(ℝX^{0}) with X^{0} = iZ^{0}, and N^{−} = P^{−}∩G^{c}. We let N^{+} = P^{+} ∩ G^{c} = τ(N^{−}) = θ(N^{−}) We remark that things are set up so that

Ω = H/H ∩ K = HP_{max}/P_{max} .

Let us recall some facts about parabolic induction that will be used in this and next section. Let 𝔞_{q} be the maximal abelian subspace in 𝔮 ∩ 𝔨 from Theorem 2.6. Let 𝔞^{c} := i𝔞 ⊂ 𝔮^{c} ∩ 𝔭^{c}. Then 𝔞^{c} is maximal abelian in 𝔮^{c} ∩ 𝔭^{c}. Let Δ^{c} be the roots of 𝔞^{c} in 𝔤^{c} and choose a set of positive roots Δ^{c+}. Let 𝔪 = 𝔷_{𝔨}(𝔞^{c}) = 𝔷_{𝔥}(𝔞^{c}), and let 𝔫_{min} = ⊕_{α∈Δ^{c+}}𝔤_{α}^{c}. Then 𝔭_{min} = 𝔭_{min}(Δ^{c+}) = 𝔪 ⊕ 𝔞^{c} ⊕ 𝔫_{min} is a minimal parabolic subalgebra of 𝔤^{c}. Let 𝔭 = 𝔩 ⊕ 𝔞 ⊕ 𝔫 be any parabolic subalgebra containing 𝔭_{min} and such that 𝔞 ⊂ 𝔞^{c}. Let P_{min} = MAN = N_{G}(𝔭_{min}) ⊂ P =: LAN = N_{G}(𝔭). Notice that we are using 𝔭 for a moment in two different ways, but the meaning should be clear in each case. Define ρ(𝔫) ∈ 𝔞^{∗} by 2ρ(𝔫)(X) := Tr(ad(X))|𝔫, X ∈ 𝔞. Let λ ∈ 𝔞^{∗} and define

π_{λ} = Ind_{LAN}^{G^{c}}1 ⊗ −λ ⊗ 1 .

The minus sign has been inserted in order to simplify some formulas later on. The Hilbert space H(λ) for π_{λ} is the space of measurable function f : G^{c} → ℂ such that

f(glan) = a^{λ−ρ}f(g)

and ∫_{K} |f(k)|^{2} dk < ∞. The inner product in this space is (f, g) = ∫_{K} f(k)\overline{g(k)} dk. Let P̄ = τ(P) = LAN̄. Define π̄_{−λ} = IndG^{c} ; P̄1 ⊗ λ ⊗ 1 and H̄(−λ) in the same way by replacing P by P̄. The following result is then obtained by a simple calculation.

Lemma 4.2. — The map T : H̄(−λ) → H(λ), f ↦ f ∘ τ defines a G^{c}-isomorphism between π̄_{−λ} and π_{λ} ∘ τ.

We will now specialize to the situation where P = P_{max}. The set of positive roots is now chosen such that −Δ_{n}^{+} ⊂ Δ^{c+} .

Lemma 4.3 (Matsuki). — The set HP_{max} is open in G^{c}.

We also have:

Lemma 4.4. — If P is any parabolic subgroup P_{min} ⊂ P ⊂ P_{max}, then HP = HP_{max}.

Let K_{0}(λ) ⊂ H(λ) be the space of smooth functions with Supp(f) ⊂ HP_{max} such that f|H has compact support. Let

S(Ω) = {g ∈ G^{c} | g^{−1}HP_{max} ⊂ HP_{max}} = S(Ω_{ℂ}) ∩ G^{c}.

The cone C_{max} = 𝔤^{c} ∩ iW_{max} is a maximal H-invariant cone in 𝔮^{c} = i𝔮 and S(Ω) = H exp(C_{max}) = S(C_{max}). Notice that

τ(S(Ω)) = S(−C_{max}) = {g ∈ G^{c} | gHP_{max} ⊂ HP_{max}}
= {g ∈ G^{c} | g · Ω ⊂ Ω} .

Let f ∈ K_{0}(λ) and s ∈ S(−C_{max}). Then π_{λ}(s)f ∈ K_{0}(λ). Write

x = h(x)a_{H}(x)n(x) ∈ HP_{max} .

Then

π_{λ}(s)f(h) = a_{H}(s^{−1}h)^{λ−ρ}f(h(s^{−1}h)) .

We have HP_{max} ⊂ N^{+}P_{max}. For x ∈ N^{+}P_{max} write x = n̄(x)l(x)aN̄(x)nN̄(x). As X^{0} = iZ^{0} and Z_{G_{ℂ}}(Z^{0}) = K_{ℂ} it follows that our group A is just exp(i𝔷_{𝔨}) and hence aN̄(x)^{μ} = χ_{μ}(k_{c}(x)) = D_{μ}(x). We will therefore use the notation D_{μ} in the what follows. Assume that we have chosen λ such that the unitary highest weight representation ρ_{λ+ρ} exists. Then Ω×Ω ∋ (x, y) ↦ K_{λ+ρ}(x, y) ∈ ℂ is positive definite. By equation (2.2) and (2.5) we get

(4.1)  aN̄(τ(exp x)^{−1} exp y)^{−λ−ρ} = K_{λ+ρ}(x, y) x, y ∈ Ω .

Define a map L_{λ} : K_{0}(λ) → H(−λ) by

L_{λ}f(x) = ∫_{N^{+}} f(τ(x)n̄) dn̄
(4.2)  = ∫_{Ω} f(exp y) K_{λ+ρ}(y, x) dy
(4.3)  = ∫_{H/H∩K} f(h)aN̄(x^{−1}h)^{−λ−ρ} dh .

Lemma 4.5. — Let f ∈ K_{0}(λ) and s ∈ S(Ω). Then L_{λ}(π_{λ}(τ(s))f) = π_{−λ}(s)L_{λ}f.

Proof. — We have with s^{∗} = τ(s^{−1}):

L_{λ}(π_{λ}(τ(s))f)(x) = ∫_{N^{+}} f(s^{∗}y)aN̄(τ(x)^{−1}y)^{−λ−ρ} dy
= ∫_{N^{+}} aN̄(s^{∗}y)^{λ−ρ}f(n̄(s^{∗}y))aN̄(τ(x)^{−1}y)^{−λ−ρ} dy

Notice that y = τ(s)s^{∗}y = τ(s)n̄(s^{∗}y)l(s^{∗}y)aN̄(s^{∗}y)nN̄(s^{∗}y). Hence the last integral becomes

L_{λ}(π_{λ}(s^{∗})f)(x) = ∫_{N^{+}} f(n̄(s^{∗}y))aN̄(τ(x)^{−1}τ(s)n̄(s^{∗}y))^{−λ−ρ}aN̄(s^{∗}y)^{−2ρ} dy
= ∫_{N^{+}} f(n̄) aN̄(τ(s^{−1}x)^{−1}n̄)^{−λ−ρ} dn̄
= π_{−λ}(s)L_{λ}f(x) .

Motivated by the fact that the pairing

H(λ) × H(−\overline{λ}) ∋ (f, g) ↦ ∫_{K} f(k)\overline{g(k)} dk ∈ ℂ

is G^{c}-invariant, we now define a new form on K_{0}(λ) by

(f, g)_{λ} = (f, L_{λ}g) .

Lemma 4.6. — Let the notation be as above. Let f, g ∈ K_{0}(λ) and let λ ∈ 𝔞^{∗}. Define F ∘ exp = f and G ∘ exp = g. Then

(f, g)_{λ} = ∫_{Ω} ∫_{Ω} F (x)\overline{G(y)} K_{λ+ρ}(y, x) dydx
= ∫_{H_{o}} ∫_{H_{o}} f(h)\overline{g(k)}aN̄(h^{−1}k)^{−λ−ρ} dkdh.

In particular (·, ·)_{λ} is positive semidefinite if λ + ρ < a(0).

Proof. — Let F and G be as before. As Supp(F), Supp(G) ⊂ Ω it follows by equation (4.2) that

(f, g)_{λ} = ∫_{Ω} f(exp(x))\overline{L_{λ}g(exp(x))} dx
= ∫_{Ω} F (x)\overline{∫_{Ω} G(y)K_{λ+ρ}(y, x) dy} dx
= ∫_{Ω} ∫_{Ω} F (x)\overline{G(y)}K_{λ+ρ}(y, x) dydx .

The second equation follows by equation (4.3).

Assume that (·, ·)_{λ} is positive semidefinite. Let

L(λ) = {u ∈ K_{0}(λ) | ∀v ∈ K_{0}(λ) : (u, v)_{λ} = 0}

and let K(λ) be the completion of K_{0}(λ)/L(λ). Then π_{λ} defines a involutive representation of S(−C_{max}) on K(λ):

Lemma 4.7. — Let f, g ∈ K(λ). Then for all s ∈ S(−C_{max}) the relation

(π_{λ}(s)f, g)_{λ} = (f, π_{λ}(s^{∗})g)_{λ}

follows. Thus π_{λ} ∘ τ defines a involutive representation of S(Ω). In particular π_{λ}(h) is unitary for all h ∈ H and π_{λ}(exp X) is self adjoint for all X ∈ −C_{max}.

Proof. — As (π_{λ}(s)f, g) = (f, π_{−\overline{λ}}(s^{−1})g) for all λ ∈ 𝔞_{ℂ}^{∗} and λ is real it follows that

(π_{λ}(s)f, L_{λ}g) = (f, π_{−λ}(s^{−1})L_{λ}g) = (f, L_{λ}(π_{λ}(τ(s^{−1}))g) = (f, π_{λ}(s^{∗})g)_{λ} .

This lemma implies that by starting from the representation π_{λ}, we have constructed a involutive representation of the semigroup S(Ω). By the Lüscher-Mack theorem such a representation can be extended to a unitary representation of G. Another way of obtaining such an extension is to use the theory of local representation developed by P. Jorgensen [28, 29]. We will give a short review of the Lüscher-Mack theory and use reference [21, 14, 32, 39, 60, 30, 31]

We have seen that π ≃ π_{λ} ∘ τ passes to a representation on K(λ) (also denoted by π) such that π(h) is unitary and π(exp(X)), X ∈ C_{max} is self-adjoint. As a result we arrive at self-adjoint operators dπ(Y) with spectrum in (−∞, 0] such that for Y ∈ C_{max}, π(exp Y) = e^{dπ(Y)} on K(λ). As a consequence of that we notice that

t ⟼ e^{t dπ(Y)}

extends to a continuous map on {z ∈ ℂ | Re(z) ≥ 0} which is holomorphic on the open right half plane {z ∈ ℂ | Re(z) > 0}. Furthermore,

e^{(z+w) dπ(Y)} = e^{z dπ(Y)}e^{w dπ(Y)} .

The Lüscher-Mack Theorem now states, see [39]:

Theorem 4.8 (Lüscher-Mack). — Let C ⊂ 𝔮^{c} be a closed H-invariant pointed and generating convex cone. Let ρ be a strongly continuous contractive representation of S(C) on the Hilbert space H such that ρ(s)^{∗} = ρ(s^{∗}). Let G̃ be the connected, simply connected Lie group with Lie algebra 𝔤 = 𝔥⊕𝔮. Then there exists a continuous unitary representation ρ̃: G̃ → U(H), extending ρ, such that for the differentiated representations dρ and dρ̃:

1) dρ̃(X) = dρ(X) ∀X ∈ 𝔥.

2) dρ̃(iY) = i dρ(Y) ∀Y ∈ C.

We apply this to our situation and get the following theorem:

Theorem 4.9. — Let the notation be as before and assume that ρ_{λ+ρ} is unitary with λ + ρ < a(0). Then the following holds:

1) S(C) acts via s ↦ π(s) by contractions on K(λ).

2) Let G̃ be the simply connected Lie group with Lie algebra 𝔤. Then there exists a unitary representation π̃ of G̃ such that dπ̃(X) = dπ(X) for X ∈ 𝔥 and i dπ(Y) = dπ̃(iY) for Y ∈ C.

3) The representation π̃ is irreducible if and only if π is irreducible.

We notice the following consequence of 2.8:

Lemma 4.10. — Let the notation be as before. In particular assume that λ + ρ < a(0). Then π̃ is a direct integral of highest weight representations.

We will now identify the representation π̃ using the special situation we have here. Let 1_{λ+ρ} be the constant function z ↦ c_{λ+ρ} where the constant c_{λ+ρ} is determined by

||1_{λ+ρ}|| = 1 .

Define a map U : K_{0}(λ) → H(ρ_{λ+ρ}) by

Uf(z) = ∫_{H_{o}} f(h) ρ_{λ+ρ}(h)1_{λ+ρ}(z) dh
= ∫_{H_{o}/H_{o}∩K} f(h) ρ_{λ+ρ}(h)1_{λ+ρ}(z) dh
= ∫_{H_{o}} f(h) J_{λ+ρ}(h^{−1}, z)^{−1} dh
= ∫ F (h · 0)J_{λ+ρ}(h^{−1}, z)^{−1} dh

where F (h · 0) = aN̄(h)^{λ−ρ}f(n̄(h)). Notice the similarity with the map R^{∗} from equation (3.4). So in some sense the map U is just R^{∗} and hence naturally related to the generalized Segal-Bargmann transform. We refer now to [31] for the proof of the following Lemma.

Lemma 4.11. — Let f, g ∈ K_{0}(λ) then (f, g)_{λ} = (Uf, Ug).

Theorem 4.12. — The map U extends to an unitary isomorphism U : K(λ) → H(ρ_{λ+ρ}) such that U(π_{λ}(τ(s))f) = ρ_{λ+ρ}(s)U(f), s ∈ S(Ω). In particular { \atop (π_{λ} ∘ τ)} is unitarily isomorphic to ρ_{λ+ρ}

Proof. — We will prove here the intertwining relation U(π_{λ}(τ(g))f) = ρ_{λ+ρ}(g)U(f). For that we need the following transformation rule for the integral over H (see [30], Lemma 5.12):

∫_{H_{o}/H_{o}∩K} f(h(sh)H ∩ K)a_{H}(sh)^{−2ρ} dḣ = ∫_{H_{o}/H_{o}∩K} f(hH ∩ K) dḣ .

We also notice that if X ∈ 𝔭^{+} then

ρ_{λ+ρ}(X)1_{λ+ρ} = 0 .

Thus if we decompose s^{∗}h as s^{∗}h = h_{opp}(s^{∗}h)a_{H,opp}(s^{∗}h)n̄ ∈ HAN̄, then

ρ_{λ}(s^{∗}h)1_{λ+ρ} = a_{H,opp}(s^{∗}h)^{λ+ρ}ρ_{λ+ρ}(s^{∗}h)1_{λ+ρ} .

Finally the relation between our usual decomposition according to HAN and the one using HAN̄, denoted by the subscript _{opp} in the following, is

x = h(x)a_{H}(x)n ⟺ τ(x) = h(x)a_{H}(x)^{−1}τ(n) = h_{opp}(τ(x))a_{H,opp}(τ(x))n̄ .

Thus

h(x) = h_{opp}(x) and a_{H,opp}(τ(x)) = a_{H}(x)^{−1} .

Using this we get

U(π_{λ}(τ(s))f)(z) = ∫_{H_{o}} f(s^{∗}h) ρ_{λ+ρ}(h)1_{λ+ρ} dh
= ∫_{H_{o}} f(h(s^{∗}h))a_{H}(s^{∗}h)^{λ−ρ}ρ_{λ+ρ}(h)1_{λ+ρ} dh
= ∫_{H_{o}} f(h(s^{∗}h))a_{H}(s^{∗}h)^{−2ρ} ρ_{λ+ρ}(sh(s^{∗}h))1_{λ+ρ} dh
= ∫_{H_{o}} f(h) ρ_{λ+ρ}(sh)1_{λ+ρ} dh
= ρ_{λ+ρ}(s)Uf(z) .

As this holds for all s ∈ S(Ω) it follows that π̃(s) = ρ_{λ+ρ}(s) for all s ∈ S(Ω) = S(Ω_{ℂ}) ∩ G^{c}. But then both of them have to agree on S(W (ρ_{λ+ρ})).

Our inner product for the realization of ρ_{λ+ρ} has some peculiar properties. Let U ⊂ Ω be open and let K_{0,U}(λ) be the set of functions in K_{0}(λ) such that Supp(f) ⊂ U and notice that we can make U arbitrary small. Let K_{U}(λ) be the projection of K_{0,U}(λ) into K(λ).

Lemma 4.13. — Let U ≠ ∅ be an open set in Ω. Then K_{U}(λ) is dense in K(λ).

The argument proving Lemma 4.13 actually shows that the δ-distribution f ↦ f(0) is in K(λ). Take δ in the definition of U to get

U(δ)(z) = 1_{λ+ρ}(z)

or

U^{∗}1_{λ+ρ} = δ .

Now an orthogonal basis of H(ρ_{λ+ρ}) can be constructed applying elements of U(𝔭^{−}) to 1_{λ+ρ}. This corresponds to applying differential operators to 1_{λ+ρ}. The above arguments imply the following lemma:

Lemma 4.14. — The δ distribution is a normalized lowest K-type of the highest weight moduleK(λ) corresponding to the constant function z ↦ c_{λ+ρ}. The K-finite elements of K(λ) are finite linear combinations of derivatives of the δ -distribution.

We end this section by pointing out one difference in our presentation here and the one in [31]. In that paper we identified Ω with a bounded subset in 𝔭^{−} ∩ 𝔤^{c} instead of 𝔭^{+} ∩ 𝔤^{c}. The action of G^{c} in those two realization is just a twist by τ. Therefore the action of the group on Ω is twisted by τ but there is no twist in the identification of the representation, expressed here by the fact that π_{λ} ∘ τ ≃ π̄_{−λ}.

5. The Character Formula

We will now show that the relation between the representations π_{λ} ∘ τ and ρ_{λ+ρ} can also been made precise using the H-invariant spherical distribution characters. For a moment we will let G stand for any semisimple Lie group and H for a closed subgroup. Let π be a unitary representation of G or one of the induced representations π_{λ} defined in the last section. The space H(π)^{∞} can be made into a complete locally convex, and Hausdorff topological vector space, and G acts continuously on H(π)^{∞} by π^{∞}(g) = π(g)|H(π)^{∞}, see [66]. Let H(π)^{−∞} be the continuous dual of H(π)^{∞}. Then G acts on H(π)^{−∞} by

< u, π^{−∞}(g)ν >:=< π^{∞}(g^{−1})u, ν > , u ∈ H(π)^{∞}, ν ∈ H(π)^{−∞} .

Here < ·, · > stands for the canonical bilinear pairing between H(π)^{∞} and H(π)^{−∞}. A non-zero element ν ∈ H(π)^{−∞H} is called an H-invariant distribution vector. Let ν be an H-invariant distribution vector. Then we can imbed H(π)^{∞} into C^{∞}(G/H) by

H(π)^{∞} ∋ u ↦ (gH ↦< π(g^{−1})u, ν >) ∈ C^{∞}(G/H) .

Let ν ∈ H(π)^{−∞H} and φ ∈ C_{c}^{∞}(G/H) and define π^{−∞}(φ)ν by

π^{−∞}(φ)ν = ∫_{G/H} φ(ġ) π^{−∞}(g)ν dġ .

Notice that if π is unitary then we have a conjugate linear embedding H(π) ≃ H(π)^{∗} ⊂ H(π)^{−∞}, and π has an H-invariant distribution vector if and only if the dual representation on H(π)^{∗} has an H-invariant distribution vector, say ν^{∗}. Let ν ∈ H(π)^{−∞H}, then π^{−∞}(φ)ν ∈ H(π)^{∗∞} for all φ ∈ C_{c}^{∞}(G/H) and we can define the H-invariant distribution Θ_{π} by

C_{c}^{∞}(G/H) ∋ φ ↦ Θ_{π}(φ) :=< π^{−∞}(φ)ν, ν^{∗} >∈ ℂ .

It is in most cases an unsolved problem to find an explicit formula for the distribution Θ_{π}.

As an example let us assume that G = H × H. In this case G/H ≃ H, (a, b)H ↦ ab^{−1}, such that the action of H × H on G/H is mapped into (a, b) · c = acb^{−1}. An irreducible unitary representation π of G has an H-invariant distribution vector if and only if π = ρ ⊗ ρ̄, where ρ is an irreducible unitary representation of H and ρ̄ is the representation on H̄(ρ) ≃ H(ρ)^{∗}. In this case there is – up to scalar – only one H-invariant distribution vector, and the corresponding distribution is a multiple of the usual character

Θ_{π}(φ) = Tr π(φ) .

As a second example we take one of the representations π_{−λ} = Ind_{P_{min}}^{G^{c}}1 ⊗ λ ⊗ 1 where P_{min} = MAN is a minimal parabolic subgroup of G^{c}. Define the Weyl groups W = N_{K}(A)/Z_{K}(A) and W_{0} = N_{K∩H}(A)/Z_{K∩H}(A). Then W_{0} ⊂ W and W_{0}\W = ˙∪_{j=1}^{r}W_{0}m_{j}, for some elements 1 = m_{1}, …, m_{r} ∈ W. Then by [40] it follows that

˙⋃_{j=1}^{r}Hm_{j}MAN ⊂ G^{c}

is a open and dense. Associated to each of those open orbit is an H -invariant distribution vector [45]. We will describe the construction for the set HP_{min}. For x ∈ HP_{min} write

x = h(x)b_{H}(x)n(x) ∈ HAN .

Define p_{λ}(x) = b_{H}(x)^{−λ−ρ}, λ ∈ 𝔞_{ℂ}^{c∗}, for x ∈ HAN, and p_{λ}(x) = 0 for x ∉ HAN. The function p_{λ} is sometimes called the Poission kernel of G^{c}/H (related to the open orbit HP_{min}). We have p_{λ}(xman) = a^{−λ−ρ}p_{λ}(x), and by [45] it is known that p_{λ} is continuous if Re < λ + ρ, α >< 0 for all positive roots. In that case p_{−λ} ∈ H(λ) ↪ H(−λ)^{−∞}. Here the pairing is given by

< f, p_{−λ} >= ∫_{K} f(k)p_{−λ}(k) dk = ∫_{H} f(h) dh , f ∈ H(−λ)^{∞} .

By construction it follows that p_{−λ} is H-invariant, or p_{−λ} ∈ H(−λ)^{−∞H}. Again by [45] it follows that λ ↦ p_{−λ} ∈ H(−λ)^{−∞H} has a meromorphic continuation to all of 𝔞_{ℂ}^{c∗}. If φ ∈ C_{c}^{∞}(G^{c}/H) then π_{−λ}^{−∞}(φ)p_{−λ} ∈ H(λ)^{∞}. Hence we can apply the distribution vector p_{λ} to π_{−λ}^{−∞}(φ)p_{−λ} to get the distribution ζ_{λ} defined by

< φ, ζ_{λ} >=< π_{−λ}^{−∞}(φ)p_{−λ}, p_{λ} > .

Formally, without bothering about convergence or the use of Fubini’s Theorem, we get

< φ, ζ_{λ} > = ∫_{H} π_{−λ}^{−∞}(φ)p_{−λ}(h) dh
= ∫_{H} ∫_{G^{c}/H} φ(ġ)p_{−λ}(g^{−1}h) dgdh
= ∫_{G^{c}/H} φ(ġ) [∫_{H} p_{−λ}(g^{−1}h) dh] dġ
= ∫_{G^{c}/H} φ(ġ)φ_{λ}(ġ^{−1}h) dġ .

It turns out that this calculation is not only formal for our spaces G^{c}/H and certain spectral parameters λ, but it can be made formally correct and gives rise to the theory of spherical functions, see [11, 48].

We will now assume that ρ_{π} is an unitary highest weight module corresponding to a holomorphic discrete series of G/H. (The same arguments also holds for the universal covering space G̃/H, but the arguments become more involved because again we would have to discuss the universal covering of the semigroups S(W (π)), the lifing of g ↦ k(g) to G̃ and K̃, etc.) We refer to [4, 24, 35, 36, 51, 52, 55] for the theory of spherical highest weight modules, Hardy spaces and the holomorphic discrete series. We use the notation from section 2. In particular 𝔱 is a Cartan subalgebra of 𝔨 and 𝔤 such that 𝔞_{q} = 𝔱 ∩ 𝔮 is maximal abelian in 𝔮. We use Δ, etc for the set of roots of 𝔱_{ℂ} in 𝔤_{ℂ}, whereas Δ̃ stands for the set of roots of 𝔞_{qℂ} in 𝔤_{ℂ}. We choose compatible sets of roots, i.e., Δ̃^{+} = {α̃ = α|𝔞_{q} | α ∈ Δ}. In this case ρ̃ = ρ|𝔞_{q}.

Let W be a G-invariant cone in 𝔤 such that W_{min} ⊂ W ⊂ W_{max} for some choice of ordering. Let S(W) = G exp(iW), S(W^{o}) = G exp(iW^{o}) = S(W)^{o}, and define Ξ ⊂ G_{ℂ}/H_{ℂ} by

Ξ = Ξ(W^{o}) = S(−W^{o})x_{o}, x_{o} = {H_{ℂ}} ∈ G_{ℂ}/H_{ℂ} .

Then Ξ is an open submanifold of G_{ℂ}/H_{ℂ}. In particular Ξ is complex. We notice that G/H ⊂ cl(Ξ)\Ξ and that γ^{−1}(G/H) ⊂ Ξ for all γ ∈ S(W^{o}). Let H_{2}(W) be the Hardy space of all holomorphic functions on Ξ such that γ · F : G/H ∋ m ↦ F (γ^{−1}m) ∈ ℂ, is in L^{2}(G/H) and the L^{2}-limit

β(F) := lim_{γ→1} γ · F ∈ L^{2}(G/H)

exists. Define an inner product on H_{2}(W) by

(F, G) = (β(F), β(G)) .

Then H_{2}(W) is a Hilbert space carrying an involutive representation of S(W) defined by γ · F (z) = T (γ)F (z) = F (γ^{−1} · z). This representation decomposes discretely and with multiplicity one into a direct sum of holomorphic discrete series representations

H_{2}(W) = ⨁_{π} E(π)

where the sum is taken over all the holomorphic discrete series representations that extend to holomorphic representations of S(W). Here E(π) stands for the realization of ρ_{π} in H_{2}(W). This realization can be made explicit in the following way. Let 0 ≠ u_{0} be a H ∩ K-invariant vector in V_{π}, the representation space for the minimal K-type π. Let s ∈ S(W). Then s ∈ H_{ℂ}K_{ℂ}P^{+}. Write s = hk_{H}(s)p. For u ∈ V_{π} let

φ_{π,u}(s) := (π(k_{H}(s^{−1}))u, u_{0}) .

Then φ_{π,u} ∈ H_{2}(W) if and only if < μ(π)+ρ̃, α̃ >< 0 for all α̃ ∈ Δ̃_{n}^{+} and in that case the closed G-invariant subspace generated by {φ_{π,u} | u ∈ V_{π}} is isomorphic to E(π). We normalize u_{0} so that this map is an unitary isomorphism of V_{π} into L^{2}(G/H). Notice that the map V_{π} ∋ u ↦ φ_{π,u} is K-equivariant.

The space H_{2}(W) is a reproducing Hilbert space. Let K(z, w) be the corresponding reproducing kernel. For z ∈ Ξ the map w ↦ K(w, z) = K_{z}(w) extends to a smooth function on S(−W)x_{o}. In particular K(z, ·) = β(K_{z}) is a well defined, smooth L^{2}- function on G/H. For F ∈ H_{2}(W) and γ ∈ S(−W^{o}) we have

F (γ · z) = (F, K_{γ·z})
= [T (γ^{−1})F](z)
= (T (γ^{−1})F, K_{z})
= (F, T (\overline{γ})K_{z}) .

Thus

(5.1)  K(γ · z, w) = K(z, γ^{∗} · w) .

It follows that K is determined by the function k ∈ O(Ξ)^{H}, k(z) = K(z, x_{0}), and K(γ_{1} · x_{0}, γ_{2} · x_{0}) = k(γ_{2}^{∗}γ_{1}) (by abuse of notation, viewing k as a H-biinvariant function). Finally we have

β^{−1}(f)(z) = F (z) = ∫_{G/H} f(m) K(z, m) dm
= (f, K_{z})
= ∫_{G/H} f(ġ) k(ġ^{−1} · z) dġ

Lemma 5.1. — Let F ∈ H_{2}(W)^{∞}. Then

(5.2)  β(F)(m) = lim_{0<t→0} F (exp(−tiZ^{0}) · m) .

Proof. — Decompose f into K-types f = ∑_{δ∈K̂} f_{δ}. The Fourier series ∑ f_{δ} converges to f in the C^{∞} -topology, a fact due to Harish-Chandra, see [66], Lemma 4.4.2.1. Denote the highest weight of δ by μ(δ). Then, using the notation from Theorem 2.3, we have μ(δ) = μ_{0} + r_{δ}λ_{0}. Each μ(δ) is a weight in one of the spaces E(π) and we have r_{π} < 0. Furthermore – as E(π)_{K} = U(𝔭^{−})V_{π} – we get

(5.3)  μ(δ) = μ(π) − ∑_{α∈Δ^{+}} n_{α}α ,  n_{α} ∈ ℕ_{0} = ℕ ∪ {0} .

Hence r_{δ} ≤ r_{π} < 0. Furthermore

δ(exp(itZ^{0})) = e^{r_{δ}t} Id .

Notice that 0 < e^{r_{δ}t} < 1 for all t > 0. Hence we have for z ∈ S(−W) · x_{0}:

f(exp(−itZ^{0}) · z) = ∑ f_{δ}(exp(−itZ^{0}) · z)
= ∑ e^{r_{δ}t}f_{δ}(z) .

It follows that the series ∑_{δ} |f_{δ}(exp(−itZ^{0}) · m)| is uniformly dominated by the series ∑_{δ} |f_{δ}(z)|, and the claim follows.

Let pr_{π} : L^{2}(G/H) → E(π) be the orthogonal projection and define T_{π} := β^{−1} ∘ pr_{π} : L^{2}(G/H) → H_{2}(W). Then

T_{π}(f)(w) := β^{−1}(pr_{π}(f))(w) = ∫ pr_{π}(f)K(w, m) dm .

As f ↦ T_{π}f(w) is continuous it comes from a function Θ_{π}(·, w) ∈ E(π) ⊂ H_{2}(W). Thus

(5.4)  β^{−1}(pr_{π}(f))(w) = ∫ f(m)\overline{Θ_{π}(m, w)} dm .

The function m ↦ Θ_{π}(m, w) extends to Ξ such that Θ_{π}(z, w) is holomorphic in the first variable, anti-holomorphic in the second variable, and Θ_{π}(z, w) = \overline{Θ_{π}(w, z)}. As the projection L^{2}(G/H) → H_{2}(W) is given by f ↦ (w ↦ (f, K_{w})) and H_{2}(W) = ⊕_{π}E(π) it follows that

K(z, w) = ∑_{π} Θ_{π}(z, w) .

Lemma 5.2. — Let {φ_{ν}}_{ν∈ℕ} be an orthonormal basis for E(π). Then Θ_{π}(z, w) = ∑_{ν=1}^{∞} φ_{ν}(z)\overline{φ_{ν}(w)} .

Proof. — This is well known, but let us recall the proof here. As z ↦ Θ_{π}(z, w) is in E(π) it follows that

Θ_{π}(z, w) = ∑ a_{ν}(w)φ_{ν}(z) .

But

φ_{ν}(w) = T_{π}(βφ_{ν})(w) = (φ_{ν}, Θ_{π}(·, w)) = \overline{a_{ν}(w)} .

Since pr_{π} and β^{−1} are intertwining it follows that

Θ_{π}(s · z, w) = Θ_{π}(z, s^{∗} · w) , s ∈ S(−W) .

Hence Θ_{π} is determined by a function θ_{π}:

(5.5)  θ_{π}(z) = Θ_{π}(z, x_{0}) = \overline{Θ_{π}(x_{0}, z)} .

By construction we have Θ_{π}(γ_{1} · x_{0}, γ_{2} · x_{0}) = θ_{π}(γ_{2}^{∗}γ_{1}) and

T_{π}f(γ · x_{0}) = ∫_{G/H} f(ġ)θ_{π}(ġ^{−1}γ) dġ = ∫_{G/H} f(m) \overline{θ_{π}(γ^{∗}m)} dm .

Furthermore we have the Lemma:

Lemma 5.3. — The reproducing kernel has a decomposition in the form k = ∑_{π} θ_{π}.

Let Θ_{π} be the spherical distribution defined by f ↦ pr_{π}(f)(x_{0}). We can now realize the spherical distribution character as a hyperfunction on G/H in the following way:

Lemma 5.4. — Let f ∈ C_{c}^{∞}(G/H). Then

Θ_{π}(f) = lim_{0<t→0} ∫_{G/H} f(m)Θ_{π}(exp(−tiZ^{0}) · x_{0}, m) dm
= lim_{0<t→0} ∫_{G/H} f(m)\overline{θ_{π}(exp(−tiZ^{0}) · m)} dm

Furthermore we have

1. The distribution Θ_{π} extends to a holomorphic function on Ξ = S(−W^{o}) · x_{0} given by Θ_{π}(z) = θ_{π}(z) = ∑ \overline{φ_{ν}(x_{0})}φ_{ν}(z);

2. There exists a character χ : 𝔻(G/H) → ℂ such that DΘ_{π} = χ(D)Θ_{π}.

Proof. — It is clear that Θ_{π} is an H-invariant distribution. The first part now follows by (5.2), (5.4), (5.5), and Lemma 5.2. Let D ∈ 𝔻(G/H). Then D : βE(π)_{K} → L^{2}(G/H). But the multiplicity of ρ_{π} in L^{2}(G/H) is one. Hence D(βE(π)_{K}) = E(π)_{K}. But E(π) is irreducible, so D has to be a scalar on E(π).

Finding the character formula for the holomorphic discrete series now becomes the problem of determining the function θ_{π}. Let μ = μ(π) denote the highest weight of π, let u = u_{μ(π)} ∈ V_{π} be a highest weight vector. We assume that φ_{1}(z) = φ_{π,u}. Furthermore ∥u∥ = 1 as ∥φ_{1}∥ = 1. We have

φ_{1}(t^{−1}z) = t^{μ}φ_{1}(z) , ∀t ∈ exp(𝔱_{ℂ}) ∩ S(W) .

We assume that the other functions φ_{ν} are weight vectors of weight say μ_{ν}. Since H(ρ_{π}) = U(𝔭^{−}) · V_{π} it follows that

μ_{ν} = μ − ∑_{α∈Δ^{+}} n_{α}α , n_{α} ∈ ℕ ∪ {0} =: ℕ_{0} .

This, φ_{1}(x_{0}) = (u, u_{0}), and Lemma 5.4 imply the following.

Theorem 5.5. — Let θ_{π} = ∑ \overline{φ_{ν}(x_{0})}φ_{ν} be as above. Then θ_{π} is a H-invariant function on Ξ such that for all D ∈ 𝔻(G/H) = 𝔻(G^{c}/H)

Dθ_{π} = χ_{π}(D)θ_{π}

where χ_{π} is a character on 𝔻(G/H). Furthermore

lim_{t→∞} e^{−tμ(π)(X)}θ_{π}(exp −tX · x_{0}) = |(u, u_{0})|^{2}

for all X ∈ i𝔱, such that α(X) > 0 for all α ∈ Δ_{c}^{+} ∪ Δ_{n}^{+}.

Theorem 5.5 implies that θ_{π}|G^{c}/H∩Ξ is a H -invariant eigenfunction on S(−C^{o})/H, C = W ∩ 𝔮^{c}. We will therefore turn our attention to spherical functions on G^{c}/H. We use [11, 48] as standard reference.

Definition 5.6. — A continuous H-invariant function φ : S(−C^{o}) · x_{0} → ℂ, φ ≠ 0, is called a spherical function if there exists a character χ : 𝔻(G^{c}/H) → ℂ such that (in the sense of distributions)

Dφ = χ(D)φ , ∀D ∈ 𝔻(G^{c}/H) .

To construct spherical functions let 𝔞^{c} = i𝔞, and P_{min} = MAN be the minimal parabolic subgroup of G corresponding to the positive system Δ̃^{+}. Then N^{+} := P^{+} ∩ G^{c} ⊂ N, N = N_{0}N^{+}, and N̄ = N̄_{0}N^{−}. Let LA = K_{ℂ} ∩ G^{c} as before. Let now X^{0} = −iZ^{0}, and define Δ_{+} = {α ∈ Δ(𝔤^{c}, 𝔞^{c}) | α(X^{0}) = 1}. For α ∈ Δ_{+} let H_{α} ∈ 𝔞^{c} be determined by H_{α} = [X, τ(X)] for some X ∈ 𝔤_{α}^{c} and α(H_{α}) = 2. Let Δ_{0} = {α ∈ Δ | α(X^{0}) = 0} and Δ_{0}^{+} = Δ_{0} ∩ Δ^{+}.

Lemma 5.7. — Let λ ∈ 𝔞_{pℂ}^{∗} and s ∈ S(−C_{max}^{o}) Then the following conditions are equivalent.

1. The function H ∋ h ↦ p_{−λ}(sh) ∈ ℂ is integrable;

2. The function Ω ∋ ω ↦ p_{λ}(ω) ∈ ℂ is integrable,

3. λ ∈ E := {λ ∈ 𝔞_{pℂ}^{∗} | ∀α ∈ Δ_{+} : Re < λ, H_{α} >< 2 − m_{α}} .

The equivalence of (1) and (2) was first proved in [11]. The implication (3)⟹(2) was proved in [48], and finally (2)⟹(3) was proved in [37].

Assume that λ ∈ E. Define

φ_{λ}(s) := ∫_{H} p_{−λ}(sh) dh

and

c_{Ω}(λ) = ∫_{Ω} p_{λ}(exp X) dX .

Let c_{0}(λ) be the Harish-Chandra c-function for the Riemannian symmetric space L/H ∩ K. Thus

c_{0}(λ) = ∫_{N̄∩L} p_{λ}(n̄) dn̄

wherever the integral converges. Finally we let

c(λ) := ∫_{N̄∩HP_{min}} p_{λ}(n̄) dn̄ = c_{Ω}(λ)c_{0}(λ) .

There is a well known product formula for the c-function for a Riemannian symmetric space in terms of the c-function c_{α}(λ_{α}) for rank one symmetric spaces constructed from the roots α, see [18]. For the space G^{c}/H this was an open problem for a long time. The general case was solved by B. Krötz and the author in [37]. We refer to that article and [50] for further references.

Theorem 5.8 (KÓ99). — The following product formula for c_{Ω}(λ) is valid

(5.6)  c_{Ω}(λ) = d ∏_{α∈Δ_{+}} B (\frac{m_{α}}{2} , − < λ, \frac{1}{2}H_{α} > −\frac{m_{α}}{2} + 1)

(5.7)  = d ∏_{α∈Δ_{+}} \frac{Γ (\frac{m_{α}}{2} ) Γ (−\frac{λ(H_{α})+m_{α}}{2} + 1)}{Γ (−\frac{λ(H_{α})}{2} + 1)}

for some constante d. In particular the c-function for G/H is a product over the c-functions for the rank one subsymmetric spaces G(α)/H ∩ G(α), α ∈ Δ^{+}:

c(λ) = ∏_{α∈Δ^{+}} c_{α}(λ_{α}) .

Here G(α) is the analytic subgroup of G^{c} corresponding to the Lie algebra 𝔤(α) = 𝔤_{α} + [𝔤_{−α}, 𝔤_{α}] + 𝔤_{−α}.

For the function φ_{λ} we have according to [11, 48]:

Theorem 5.9. — The function φ_{λ} is a spherical function for λ ∈ E. It has a meromorphic continuation as a spherical function for λ ∈ 𝔞_{ℂ}^{c∗}. Furthermore

lim_{0<t→∞} e^{−t<λ−ρ̃,X>}φ_{λ}(exp(tX)) = c(λ)

for all X ∈ (−C_{max}^{o}) ∩ {Y ∈ 𝔞^{c} | ∀α ∈ Δ_{0}^{+} : α(Y) > 0} and λ ∈ E.

Recall that if π is a irreducible representation of K with highest weight μ, and such that V_{π}^{H∩K} ≠ {0}, then μ ∈ i𝔞_{q}^{∗} = 𝔞^{c∗}.

Corollary 5.10. — Let the notation be as before. In particular ρ_{π} is a holomorphic discrete series representation of G/H. Denote the highest weight of π by μ. Let λ := μ + ρ̃ ∈ 𝔞^{c∗}. Assume that λ ∈ E. Then

θ_{π}(s · x_{0}) = \frac{|(u, u_{0})|^{2}}{c(λ)} φ_{λ}(s · x_{0}) .

Define the coefficients Γ_{μ}(λ) by recursion as in [18], p. 427, and set for a ∈ exp(−C_{max}^{o} ∩ {Y ∈ 𝔞_{p} | ∀α ∈ Δ_{0}^{+} : α(Y) > 0}) :

Φ_{λ}(a) = a^{λ−ρ} ∑_{μ∈ℕ_{0}Δ^{+}} Γ_{μ}(λ)a^{−μ} .

Theorem 5.11 (Ó97). — For generic λ ∈ 𝔞_{ℂ}^{∗} we have

φ_{λ} = ∑_{w∈W_{0}} c(w · λ)Φ_{w·λ} = c_{Ω}(λ) ∑_{w∈W_{0}} c_{0}(w · λ)Φ_{w·λ}.

For the evaluation of (u, u_{0}) we recall some facts from [33]. As before we let μ = μ(π) and λ = μ + ρ̃. Let

d(μ)^{G} = \frac{∏_{α∈Δ^{+}}(μ + ρ, α)}{∏_{α∈Δ^{+}}(ρ, α)} .

If ρ_{π} is in the holomorphic discrete series of G, then d(μ)^{G} is the formal dimension of ρ_{π}, [17].

Lemma 5.12. — Let the notation be as above. Then

|(u, u_{0})|^{2} = d(μ)^{G}c(λ) .

Proof. — By definition

< φ_{1}, ρ_{π}^{−∞}(g)Θ_{π} >= φ_{1}(g · x_{0}) .

We also have ∥φ_{1}∥ = 1. Assume first that ρ_{π} belongs to the holomorphic discrete series of G and that λ ∈ E. Then |(u, u_{0})|^{2} = d(μ)^{G}c(λ) follows from Definition III.3 and Theorem III.4 in [33]. The general statement follows now by analytic continuation, see Theorem IV.15 in [33].

Using the same analytic continuation arguments as in the proof of Theorem IV.15,

[33] combined with Corollary 5.10 and Theorem 5.11 we arrive at the following character formula for the holomorphic discrete series representation ρ_{π}.

Theorem 5.13. — Assume that ρ_{π} corresponds to a holomorphic discrete series represention of G/H. Let μ be the highest weight of π and let λ = μ + ρ̃. Then

θ_{π} = d(μ)^{G}φ_{λ} .

In particular

θ_{π} = d(μ)^{G}c_{Ω}(λ) ∑_{w∈W_{0}} c_{0}(w · λ)Φ_{w·λ}

for generic μ.

This gives us the character formula for ρ_{π}. Assume now that π is a character χ_{λ+ρ} related to the representation π̄_{−λ} as before. Then this does not relate the H-spherical character of π directly with that of π̄_{−λ}. But write ρ(Δ^{+}) = ρ_{0} + ρ_{+} where ρ_{0} is the sum over the compact roots and ρ_{+} is the sum over the non-compact roots. Then H(−λ) can be embedded into H̄_{P_{min}}(−λ − ρ_{0}) where the subscript _{P_{min}} indicates that we are inducing from the minimal parabolic subgroup. It is easy to check that the restriction of p_{−λ} to H̄(−λ) is non-zero and corresponds to the corresponding Poisson kernel. Thus by restriction φ_{λ} can be viewed as the character of π̄_{−λ} (related to the open orbit HP_{min} = HP_{max}).

As a final remark we would like to mention, that most of the compactly causal symmetric spaces G/H can be compactified as the Sylov boundary G_{1}/P_{1max} of a bounded symmetric domain G_{1}/K_{1} of tube type, see [3, 55] for details and list. In short there exists a connected semisimple Lie group G_{1} such that G_{1}/K_{1} is a bounded symmetric domain which can also be realized as a tube type domain. The Sylov boundary of G_{1}/K_{1} is a compact symmetric space S_{1} = K_{1}/L_{1} = G_{1}/P_{1max}, where P_{1max} is a maximal parabolic subgroup of G_{1}. Furthermore there exists an injective G-map F : G/H → S_{1} with open dense image. Using F one can identify L^{2}(G/H) with L^{2}(S_{1}). The map F can be extended to a holomorphic isomorphism of Ξ into a open dense subset of G_{1}/K_{1}. This can be used to compare the Hardy space on G/H (or a covering in some cases) with the classical Hardy space. We refer to [4, 55] for details. It is still an open question how to use the orbit G-structure of S_{1} to analyze L^{2}(G/H).

References

[1] F. A. Berezin, Connection between co- and contravariant symbols of operators on the classical complex symmetric spaces Dokl. Akad. Nauk USSR 241 (1978) 15–17 .

[2] F. A. Berezin, F. I. Karpelevic, Zonal spherical functions and Laplace operators on some symmetric spaces, Dokl. Akad. Nauk USSR 118 (1858), 9–12 .

[3] F. Betten: Causal compactification of compactly causal spaceses. To appear in Trans. Amer. Math. Soc.

[4] F. Betten, and G. Ólafsson, Causal compactification and Hardy spaces for spaces of Hermitian type, preprint, 1998, http://www.math.math.lsu.edu/ preprint.

[5] M.G. Davidson and R.C. Fabec, Geometric realizations for highest weight representations, Representation Theory and Harmonic Analysis (Cincinnati, OH, 1994) (T. Ton-That, K.I. Gross, D. St. P. Richards and P.J. Sally, Jr., eds.), Contemporary Mathematics, vol. 191, American Mathematical Society, Providence, Rhode Island, 1995, pp. 13–31.

[6] M.G. Davidson and R.J. Stanke, Ladder representation norms for hermitian symmetric groups, to appear in Journal of Lie Theory.

[7] G. van Dijk and S.C. Hille, Maximal degenerate representations, Berezin kernels, and canonical representations, Lie Groups and Lie algebras, their Representations, Generalizations and Applications. Kluwer Academic Press, 1997 .

[8] G. van Dijk and M. Pevzner, Berezin Kernels on Tube Domains Report MI 99–34, University of Leiden .

[9] T.J. Enright, Unitary representations for two real forms of a semisimple Lie algebra: A theory of comparison, Lie Group Representations, I (College Park, Maryland, 1982/1983), Lecture Notes in Mathematics, vol. 1024, Springer, Berlin–New York, 1983, pp. 1–29.

[10] T.J. Enright, R. Howe, and N. Wallach, A classification of unitary highest weight modules, Representation Theory of Reductive Groups (Park City, Utah, 1982) (P. C. Trombi, ed.), Progress in Mathematics, vol. 40, Birkhäuser, Boston, 1983, pp. 97–143.

[11] J. Faraut, J. Hilgert, and G. Ólafsson, Spherical functions on ordered symmetric spaces, Ann. Inst. Fourier (Grenoble) 44 (1994), 927–965.

[12] J. Faraut and A. Korányi, Analysis on Symmetric Cones, Oxford Mathematical Monographs, Clarendon Press, Oxford, 1994.

[13] G.B. Folland, Harmonic Analysis in Phase Space, Annals of Mathematics Studies, vol. 122, Princeton University Press, Princeton, New Jersey, 1989.

[14] J. Fröhlich, K. Osterwalder, and E. Seiler, On virtual representations of symmetric spaces and their analytic continuation , Ann. of Math. (2) 118 (1983), 461–489.

[15] Harish-Chandra, Representations of semisimple Lie groups, IV, Amer. J. Math. 77 (1955), 743–777.

[16] Harish-Chandra, Representations of semisimple Lie groups , V, Amer. J. Math. 78 (1956), 1–41.

[17] Harish-Chandra, Representations of semisimple Lie groups , VI, Amer. J. Math. 78 (1956), 564–628.

[18] S. Helgason, Groups and Geometric Analysis: Integral Geometry, Invariant Differential Operators and Spherical Functions, Pure and Applied Mathematics, vol. 113, Academic Press, New York–London, 1984.

[19] S.C. Hille. Thesis, 1999

[20] O. Hijab, Hermite functions on compact Lie groups, I, J. Funct. Anal. 125 (1994) 480–492.

[21] J. Hilgert and K.-H. Neeb, Lie Semigroups and Their Applications, Lecture Notes in Math., vol. 1552, Springer-Verlag, Berlin, 1993.

[22] J. Hilgert and G. Ólafsson, Analytic extensions of representations, the solvable case, Japan. J. Math. (N.S.) 18 (1992) 213–289.

[23] J. Hilgert and G. Ólafsson, Causal Symmetric Spaces: Geometry and Harmonic Analysis, Perspectives in Mathematics, vol. 18, Academic Press, Inc., San Diego, California, 1997.

[24] J. Hilgert, G. Ólafsson, and B. Ørsted, Hardy spaces on affine symmetric spaces, J. Reine Angew. Math. 415 (1991), 189–218.

[25] H.A. Jaffee, Real forms of Hermitian symmetric spaces , Bull. Amer. Math. Soc. 81 (1975), 456–458.

[26] H.A. Jaffee, Anti-holomorphic automorphisms of the exceptional symmetric domains, J. Differential Geom. 13 (1978), 79–86.

[27] H.P. Jakobsen, Hermitian symmetric spaces and their unitary highest weight modules, J. Funct. Anal. 52 (1983) 385–412.

[28] P.E.T. Jorgensen, Analytic continuation of local representations of Lie groups, Pacific J. Math. 125 (1986), 397–408.

[29] P.E.T. Jorgensen, Analytic continuation of local representations of symmetric spaces, J. Funct. Anal. 70 (1987), 304–322.

[30] P.E.T. Jorgensen and G. Ólafsson, Unitary representations of Lie groups with reflection symmetry, J. Funct. Anal. 158 (1998), 26–88.

[31] P.E.T. Jorgensen and G. Ólafsson, Unitary Representations and Osterwalder–Schrader Duality The Mathematical Legacy of Harish-Chandra. A Celebration of Representation Theory and Harmonic Analysis. PSPM, 68, 2000

[32] A. Klein and L.J. Landau, From the Euclidean group to the Poincaré group via Osterwalder-Schrader positivity, Comm. Math. Phys. 87 (1982/83), 469–484.

[33] B. Krötz, Formal dimension for semisimple symmetric spaces, preprint 1998, submitted.

[34] B. Krötz and K.-H. Neeb, On hyperbolic cones and mixed symmetric spaces, J. Lie Theory 6 (1996), 69–146.

[35] B. Krötz, K.-H. Neeb, and G. Ólafsson, Spherical representations and mixed symmetric spaces, Represent. Theory 1 (1997), 424–461.

[36] B. Krötz, K.-H. Neeb, and G. Ólafsson, Spherical Functions on Mixed Symmetric Spaces, preprint 1998, submitted. http://www.math.math.lsu.edu/ preprint.

[37] B. Krötz, G. Ólafsson, The c-function for non-compactly causal symmetric spaces Submitted. Preprint 1999, http://www.math.math.lsu.edu/ preprint.

[38] J.D. Lawson, Polar and Ol’shanskii decompositions, J. Reine Angew. Math. 448 (1994), 191–219.

[39] M. Lüscher and G. Mack, Global conformal invariance in quantum field theory, Comm. Math. Phys. 41 (1975), 203–234.

[40] T. Matsuki, Orbits on affine symmetric spaces under the action of parabolic subgroups, Hiroshima Math. J. 12 (1982), 307–320.

[41] V.F. Molchanov, Quantication on para-Hermitian symmetric spaces, Adv. in Math. Sci., AMS Transl. Ser. 2, 175 (1996), 81–96

[42] K.-H. Neeb, Holomorphy and Convexity in Lie Theory, Expositions in Mathematics, de Gruyter, Berlin, 1999.

[43] Yu A. Neretin, Matrix analogs of the integral ∫_{0}^{∞} \frac{x^{α−1} dx}{(1+x)^{p}} = B(α, ρ−α) and Plancherel formula for Berezin Kernel representations. Preprint 1999

[44] A. Neumann, G. Ólafsson: Minimal and Maximal Semigroups Related to Causal Symmetric Spaces. To appear in Semigroup Forum. http://www.math.math.lsu.edu/ preprint.

[45] G. Ólafsson, Fourier and Poisson transformation associated to a semisimple symmetric space, Invent. Math. 90 (1987), 605–629.

[46] G. Ólafsson, Causal symmetric spaces, Mathematica Gottingensis 15 (1990).

[47] G. Ólafsson, Symmetric spaces of Hermitian type, Differential Geom. Appl. 1 (1991), 195–233.

[48] G. Ólafsson, Spherical functions on ordered symmetric space; A Harish-Chandra type formula, http://www.math.math.lsu.edu/ preprint.

[49] G. Ólafsson, Open problems in harmonic analysis on causal symmetric spaces, Positivity in Lie Theory: Open Problems (J. Hilgert, J. D. Lawson, K-H. Neeb, and E. B. Vinberg, eds.), Walter de Gruyter, Berlin–New York, 1998.

[50] G. Ólafsson, The c-function for a semisimple symmetric space. Preprint, January 2000. Submitted http://www.math.math.lsu.edu/ preprint.

[51] G. Ólafsson and B. Ørsted, The holomorphic discrete series for affine symmetric spaces, I, J. Funct. Anal. 81 (1988), 126–159.

[52] G. Ólafsson and B. Ørsted, The holomorphic discrete series of an affine symmetric space and representations with reproducing kernels, Trans. Amer. Math. Soc. 326 (1991), 385– 405.

[53] G. Ólafsson and B. Ørsted, Is there an orbit method for affine symmetric spaces? The Orbit Method in Representation Theory (M. Duflo, N.V. Pedersen, and M. Vergne, eds.), Birkhäuser, Boston, 1990.

[54] G. Ólafsson and B. Ørsted, Generalizations of the Bargmann transform, Lie theory and its applications in physics (Clausthal, 1995) (H.-D. Doebner, V. K. Dobrev and J. Hilgert, eds.), World Scientific, River Edge, New Jersey, 1996.

[55] G. Ólafsson and B. Ørsted, Causal compactification and Hardy spaces, Report No. 19, 1995/96, Institut Mittag-Leffler, to appear in Trans. Amer. Math. Soc.

[56] B. Ørsted and G.K. Zhang, Generalized principal series representations and tube domains, Duke Math. J. 78 (1995), 335–357.

[57] G.I. Ol’shanskii, Invariant cones in Lie algebras, Lie semigroups, and the holomorphic discrete series, Functional Anal. Appl. 15 (1982), 275–285, translated from Funktsional. Anal. i Prilozhen. 15 (1981), 53–66 (Russian).

[58] K. Osterwalder and R. Schrader, Axioms for Euclidean Green’s functions, Comm. Math. Phys. 31 (1973), 83–112.

[59] K. Osterwalder and R. Schrader, Axioms for Euclidean Green’s functions, II, Comm. Math. Phys. 42 (1975), 281–305.

[60] R. Schrader, Reflection positivity for the complementary series of SL(2n, ℂ), Publ. Res. Inst. Math. Sci. 22 (1986), 119–141.

[61] R.J. Stanton, Analytic extension of the holomorphic discrete series, Amer. J. Math. 108 (1986), 1411–1424.

[62] E.M. Stein, Harmonic Analysis: Real-variable Methods, Orthogonality, and Oscillatory Integrals, Princeton Mathematical Series, vol. 43, Princeton University Press, Princeton, New Jersey, 1993.

[63] M.H. Stone, On unbounded operators in Hilbert space, J. Indian Math. Soc. (N.S.) 15 (1951), 155–192.

[64] A. Unterberger, and H. Upmeier, The Berezin transform and invariant differential operators, Commun. Math. Phys. 164 (1994), 563–597

[65] M. Vergne and H. Rossi, Analytic continuation of the holomorphic discrete series of a semi-simple Lie group, Acta Math. 136 (1976), 1–59.

[66] G. Warner, Harmonic Analysis on Semisimple Lie Groups , I, Die Grundlehren der mathematischen Wissenschaften, Band 188, Springer-Verlag, New York–Heidelberg, 1972.

[67] N.R. Wallach, The analytic continuation of the discrete series, I, Trans. Amer. Math. Soc. 251 (1979), 1–17; II, 19–37.

[68] G. Zhang, Berezin transform on line bundles over bounded symmetric domains, Preprint 1999

Gestur Ólafsson, Department of Mathematics, Louisiana State University, Baton Rouge, LA

70803, USA