Section 1.1: A Brief Review of Vectors

Before we explore the details of orthogonal functions, let's revisit the essentials of vector analysis. Imagine two vectors in three-dimensional space.

Here, we denote the standard basis vectors as:

• i = <1, 0, 0>
• j = <0, 1, 0>
• k = <0, 0, 1>

The magnitude of a vector ||A||, also known as its norm, is determined as follows:

We define the inner product of two vectors, A and B, with the equation:

In this formula, θ represents the angle between the two vectors shown above. If two vectors are orthogonal (perpendicular), their inner product will equal zero since cos(90°) = 0.

A collection of n vectors is termed an orthogonal set when each vector is perpendicular to all others. Furthermore, if all vectors in the set have a norm of 1, we call it an orthonormal set. For instance, the set {i, j, k} = { <1, 0, 0>, <0, 1, 0>, <0, 0, 1> } is an orthonormal set, as demonstrated below.

Now that we have refreshed our memory, let’s examine how we can broaden the idea of orthogonality to functions.

Section 1.2: Introducing Orthogonal Functions

Consider a function A(x) defined over the interval a ≤ x ≤ b. One way to visualize A(x) is to think of it as a vector with an infinite number of components, where each component corresponds to the value of A(x) at a specific x within the range (a, b).

With this analogy established, we can apply concepts from vector analysis to functions. For instance, the norm of a vector is found by taking the square root of the sum of the squares of its components. We extend this concept to functions with the following definition of the norm:

Observe how this definition mirrors that of a vector's norm, differing only in that we replace the sum with an integral, given the infinite nature of the components.

Next, we define the inner product of two functions, A(x) and B(x), as follows:

Take a moment to reflect on why this definition is logical. In vector analysis, the inner product is the sum of the products of corresponding components. The formula above captures this idea and extends it to functions by substituting the sum with an integral.

To maintain the analogy, A(x) and B(x) are considered orthogonal if:

A set of functions is orthogonal if every function is orthogonal to all others. If the norms of the functions equal 1, we refer to the set as orthonormal.

Given a set { f_n(x) } of orthonormal functions:

Lastly, note that a set { f_n(x) } can be converted to an orthonormal set by dividing each function by its norm ||f_n|| if it is orthogonal but not orthonormal.

Section 2.1: Series of Orthogonal Functions

Most learners will remember the Taylor series from their early calculus studies, which illustrates how a function can be represented as an infinite weighted polynomial. Interestingly, we can express a function as an infinite series of orthogonal functions.

Consider the set { g_n(x), n ∈ N }, defined over the interval a ≤ x ≤ b, which is orthogonal relative to a weight function r(x). For a specific function f(x), we can write:

Note: We assume the series converges uniformly.

To determine the coefficients C_n, we multiply both sides by r(x)g_n(x) and integrate over the interval [a,b]:

Recalling the orthogonality property, all terms in the series vanish except when n = m, leading to:

Solving for C_n gives us:

This formulation for f(x) is referred to as an orthonormal series or a generalized Fourier series, with the coefficients C_n identified as the Fourier coefficients.

Interestingly, for a periodic function with angular frequency ω, the orthogonal functions take the form of complex exponentials: