The Margin Perceptron with Unlearning

ConstantinosPanagiotakopouloscostapan@eng.auth.grPetroulaTsampoukapetroula@gen.auth.grPhysicsDivision,SchoolofTechnology,AristotleUniversityofThessaloniki,54124Thessaloniki,Greece

Abstract

WeintroduceintotheclassicalPerceptronalgorithmwithmarginamechanismofun-learningwhichinthecourseoftheregu-larupdateallowsforareductionofpossi-blecontributionsfrom“verywellclassi ed”patternstotheweightvector.Theresultingincrementalclassi cationalgorithm,calledMarginPerceptronwithUnlearning(MPU),provablyconvergesina nitenumberofup-datestoanydesirablechosenbeforerunningapproximationofeitherthemaximalmarginortheoptimal1-normsoftmarginsolution.Moreover,anexperimentalcomparativeeval-uationinvolvingrepresentativelinearSup-portVectorMachinesrevealsthattheMPUalgorithmisverycompetitive.

1.Introduction

SupportVectorMachines(SVMs)(Boseretal.,1992;Vapnik,1998;Cristianini&Shawe-Taylor,2000)havebeenextensivelyusedaslinearclassi erseitherinthespacewherethepatternsoriginallyresideorinhighdimensionalfeaturespacesinducedbykernels.SVMsappeartobeverysuccessfulataddressingtheprob-lemofminimisinganobjectivefunctioninvolvingtheempiricalriskwhileatthesametimekeepinglowthecomplexityoftheclassi er.Asmeasuresoftheem-piricalriskvariousquantitieshavebeenproposedwiththe1-and2-normlossfunctionsbeingthemostwidelyacceptedonesgivingrisetotheoptimisationproblemsknownasL1-andL2-SVMs(Cortes&Vapnik,1995).Inthecasethatthe2-normlossfunctiontakestheplaceoftheempiricalriskanequivalentformulationexistswhichrendersthedatasetlinearlyseparableinahighdimensionalfeaturespace.Therefore,onecaneithersolvethegeneraloptimisationproblemconsist-ingoftwoterms,namelythecapacitytermandthe2-normlossorjustattempttominimisethecapacity

The Margin Perceptron with Unlearning相关文档

最新文档

返回顶部