Numerical Analysis I
AMSC/CMSC 666, Fall 2020
Course Information
Course Description

Approximation Theory.
 General overview. Leastsquares vs. the uniform norm
 Least squares I. Fourier expansions. Chebyshev polynomials
 Least squares II. The finitedimensional case: QR, SVD, PCA (LS rank approximations)
 Least squares III. Gauss quadrature
 MinMax approximations. Chebyshev interpolant. Runge effect. error estimates.

Numerical Solution of InitialValue Problems

General overview. Systems of ODEs. Existence, uniqueness & stability.

Euler's method. Multistep methods  stability and instability.
 PredictorCorrector methods:. AdamsBashforthMoulton schemes

Consistency, stability and convergence. Dahlquist theory.
 RungeKutta methods
 Local time stepping and error estimates. RK4 and RKF5.
 Stability and Convergence of RungeKutta methods

StrongStability Preserving (SSP) schemes. Methods for stiff equations
Iterative Methods for Linear Systems of Equations
 General overview. Boundaryvalue problems. Stiff systems.
 Classical methods: Jacobi, GaussSiedel, SOR, ...
 Energy functionals and gradinet methods
 Steepest descent. Conjugate gradient. ADI and dimensional splitting.
 Nonstationary methods. Local vs. global methods
 Preconditioners
 Multigrid mthods*; GMRES*
Numerical Optimization
 General overview. Fixed point iterations, loworder and Newton's methods
 Steepest descent
 Line search methods. Trust regions and Wolfe conditions
 Newton's and quasiNewton methods: Rank one/two modifications
 Rates of converegence
 Nonlinear Leastsqaures. Conjugate gradient, GaussNewton.
References
GENERAL TEXTBOOKS
K. Atkinson, An INTRODUCTION to NUMERICAL ANALYSIS, Wiley, 1987
S. Conte & C. deBoor, ELEMENTARY NUMERICAL ANALYSIS, McGrawHill
User friendly; Shows how 'it' works; Proofs, exercises and notes
G. Dahlquist & A. Bjorck, NUMERICAL METHODS, PrenticeHall,
User friendly; Shows how 'it' works; Exercises
A. Ralston & P. Rabinowitz, FIRST COURSE in
NUMERICAL ANALYSIS, 2nd ed., McGrawhill,
Detailed; Scholarly written; Comprehensive; Proofs exercises and notes
J. Stoer & R. Bulrisch, INTRODUCTION TO NUMERICAL ANALYSIS, 2nd ed., Springer
detailed account on approximation, linear solvers & eigensolvers,
ODE solvers,..
APPROXIMATION THEORY
E. W. Cheney, INTRODUCTION TO APPROXIMATION THEORY
Classical
P. Davis, INTERPOLATION & APPROXIMATION, Dover
Very readable
T. Rivlin, AN INTRODUCTION to the APPROXIMATION of FUNCTIONS
Classical
R. DeVore & G. Lorentz, CONSTRUCTIVE APPROXIMATION, Springer
A detailed account from classical theory to the modern theory; everything; Proofs exercises and notes
NUMERICAL INTEGRATION
F. Davis & P. Rabinowitz, NUMERICAL INTEGRATION,
Everything...
NUMERICAL SOLUTION Of INITIALVALUE PROBLEMS
E. Hairer, S.P. Norsett and G. Wanner, SOLVING ODEs I: NONSTIFF PROBLEMS,
SpringerVerlag, Berlin. 1991, (2nd ed)
Everything  the modern version
A. Iserles, A FIRST COURSE in the NUMERICAL ANALYSIS of DEs,
Cambridge Texts
W. Gear, NUMERICAAL INITIAL VALUE PROBLEMS in ODEs, 1971
The classical reference on theory and applications
Lambert, COMPUTATIONAL METHODS for ODEs, 1991
Detailed discussion of ideas and practical implementation
Shampine and Gordon, COMPUTER SOLUTION of ODES, 1975
Adams methods and practial implementation of ODE "black box" solvers
Butcher, NUMERICAL ANALYSIS of ODEs, 1987
Comprehensive discussion on RungeKutta methods
(mainly) ITERATIVE SOLUTION OF LINEAR SYSTEMS
A. Householder, THE THEORY OF MATRICES IN NUMERICAL ANALYSIS
The theoretical part by one of
the grand masters; Outdated in some aspects
G. H. Golub & Van Loan, MATRIX COMPUTATIONS,
The basic modern reference
Y. Saad, ITERATIVE METHODS for SPARSE LINEAR SYSTEMS,
PWS Publishing, 1996. (Available on line at
http://wwwusers.cs.umn.edu/~saad/books.html)
R. Varga, MATRIX ITERATIVE ANALYSIS,
Classical reference for the theory of iterations
James Demmel, APPLIED NUMERICAL LINEAR ALGEBRA,
SIAM, 1997
Kelley, C. T., ITERATIVE METHODS for LINEAR and NONLINEAR EQUATIONS,
SIAM 1995
NUMERICAL OPTMIZATION
J. Nocedal, S. Wright, NUMERICAL OPTIMIZATION
Springer, 1999
T. Kelly, ITERATIVE METHODS for OPTIMIZATION
SIAM
Eitan Tadmor

