On the conjugate function

[This article was first published on R-english – Freakonometrics, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

In the MAT7381 course (graduate course on regression models), we will talk about optimization, and a classical tool is the so-called conjugate. Given a function f:RpR its conjugate is function f:RpR such that f(y)=maxx{xyf(x)}so, long story short, f(y) is the maximum gap between the linear function xy and f(x).

Just to visualize, consider a simple parabolic function (in dimension 1) f(x)=x2/2, then f(2) is the maximum gap between the line x2x and function f(x).

x = seq(-100,100,length=6001) f = function(x) x^2/2 vf = Vectorize(f)(x) fstar = function(y) max(y*x-vf) vfstar = Vectorize(fstar)(x)

We can see it on the figure below.

viz = function(x0=1,YL=NA){ idx=which(abs(x)<=3) par(mfrow=c(1,2)) plot(x[idx],vf[idx],type="l",xlab="",ylab="",col="blue",lwd=2) abline(h=0,col="grey") abline(v=0,col="grey") idx2=which(x0*x>=vf) polygon(c(x[idx2],rev(x[idx2])),c(vf[idx2],rev(x0*x[idx2])),col=rgb(0,1,0,.3),border=NA) abline(a=0,b=x0,col="red") i=which.max(x0*x-vf) segments(x[i],x0*x[i],x[i],f(x[i]),lwd=3,col="red") if(is.na(YL)) YL=range(vfstar[idx]) plot(x[idx],vfstar[idx],type="l",xlab="",ylab="",col="red",lwd=1,ylim=YL) abline(h=0,col="grey") abline(v=0,col="grey") segments(x0,0,x0,fstar(x0),lwd=3,col="red") points(x0,fstar(x0),pch=19,col="red") } viz(1)

or

viz(1.5)

In that case, we can actually compute f, since f(y)=maxx{xyf(x)}=maxx{xyx2/2}The first order condition is here x=y and thusf(y)=maxx{xyx2/2}={xy(x)2/2}={y2y2/2}=y2/2And actually, that can be related to two results. The first one is to observe that f(x)=x22/2 and in that case f(y)=y22/2 from the following general result : if f(x)=xpp/p with p>1, where p denotes the standard p norm, then f(y)=yqq/q where1p+1q=1The second one is the conjugate of a quadratic function. More specifically if f(x)=xQx/2 for some definite positive matrix Qf(y)=yQ1y/2. In our case, it was a univariate problem with Q=1.

For the conjugate of the p norm, we can use the following code to visualize it

p = 3 f = function(x) abs(x)^p/p vf = Vectorize(f)(x) fstar = function(y) max(y*x-vf) vfstar = Vectorize(fstar)(x) viz(1.5)

or

p = 1.1 f = function(x) abs(x)^p/p vf = Vectorize(f)(x) fstar = function(y) max(y*x-vf) vfstar = Vectorize(fstar)(x) viz(1, YL=c(0,10))

Actually, in that case, we almost visualize that if f(x)=|x| thenf(y)={0,|y|1,|y|>1.

To conclude, another popular case, f(x)=exp(x) then{\displaystyle f^{\star}\left(y\right)={{ylog(y)y,y>00,y=0,y<0.

}}[/latex]We can visualize that case below

f = function(x) exp(x) vf = Vectorize(f)(x) fstar = function(y) max(y*x-vf) vfstar = Vectorize(fstar)(x) viz(1,YL=c(-3,3))

To leave a comment for the author, please follow the link and comment on their blog: R-english – Freakonometrics.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)