1. Properties of Proximal mapping
For a convex function h ( x ) , its proximal mapping is defined as:
P r o x h ( x ) = arg min u { h ( u ) + 1 2 ∥ u − x ∥ 2 2 } .
From the fact that the objective function is strictly convex, we know that
P r o x h ( x ) exists and is unique for all x. If
u = P r o x h ( x ) , we have
x − u ∈ ∂ h ( u ) .
1.1 L1 Lipschitz and monotone
Theorem: Proximal mapping for any convex function h ( x ) is L1 Lipschitz and monotone, that is to say
Lipschitz:∥ P r o x h ( x ) − P r o x h ( y ) ∥ 2 ≤ ∥ x − y ∥ 2 .
Monotone: ( P r o x h ( x ) − P r o x h ( y ) ) T ( x − y ) ≥ 0.
proof: From the definition of proximal mapping, if u = P r o x h ( x ) and u ^ = P r o x h ( x ^ ) , we have
x − u ∈ ∂ h ( u ) x ^ − u ^ ∈ ∂ h ( u ^ )
Then from the convexity,
h ( u ) h ( u ^ ) ≥ h ( u ^ ) + ∂ h ( u ^ ) T ( u − u ^ ) ≥ h ( u ) + ∂ h ( u ) T ( u ^ − u )
we have
( ∂ h ( u ) − ∂ h ( u ^ ) ) T ( u − u ^ ) ≥ 0 ,
which means that
( x − u − ( x ^ − u ^ ) ) T ( u − u ^ ) ≥ 0 ,
Then,
0 ≤ ∥ u − u ^ ∥ 2 2 ≤ ( x − x ^ ) T ( u − u ^ ) monotone ≤ ∥ x − x ^ ∥ 2 ∥ u − u ^ ∥ 2 Lipschitz .
1.2 Projection Property
Theorem: Proximal mapping for any convex function h ( x ) acts just like a Projection function, and its orthogonal projection is the proximal function corresponding to its conjugate function, that is to say
x = P r o x h ( x ) + P r o x h ∗ ( x ) .
proof: If u = P r o x h ( x ) , we have v = x − u ∈ ∂ h ( u ) . From the definition of conjugate function:
h ∗ ( v ) = max z { v T z − h ( z ) } = v T u − h ( u )
from which, we have
u = x − v ∈ ∂ h ∗ ( v ) , then
v = P r o x h ∗ ( x ) .
So
x = u + v = P r o x h ( x ) + P r o x h ∗ ( x ) .
1.3 Scaling and translation argument
Theorem: Let h ( x ) = f ( t x + a ) , then
P r o x h ( x ) = 1 t ( P r o x t 2 f ( t x + a ) − a )
proof: Assume
u = P r o x h ( x ) , then we have
x − u ∈ ∂ h ( u ) = t ∂ f ( t u + a )
Then
( t x + a ) − ( t u + a ) ∈ ∂ t 2 f ( t u + a )
so we have
t u + a = P r o x t 2 f ( t x + a )
so
u = P r o x h ( x ) = 1 t ( P r o x t 2 f ( t x + a ) − a ) .