Group - Past Activities

2018

  • Dec. 2018 paper accepted: our work (with Xiangyi, Sijia and Ruoyu) entitled “On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization” has been accepted by ICLR 2019; available at [openreview.net];

This paper studies a class of adaptive gradient based momentum algorithms that update the search directions and learning rates simultaneously using past gradients. We develop an analysis framework and a set of mild sufficient conditions that guarantee the convergence of the Adam-type methods, with a convergence rate of order O(log(T)/sqrt(T)) for non-convex stochastic optimization. We show that the conditions are essential, by identifying concrete examples in which violating the conditions makes an algorithm diverge. Besides providing one of the first comprehensive analysis for Adam-type methods in the non-convex setting, our results can also help the practitioners to easily monitor the progress of algorithms and determine their convergence behavior.

  • Dec. 2018 paper accepted: our work (with Xiangyi, Sijia and Ping-Yu) entitled “signSGD via Zeroth-Order Oracle” has been accepted by ICLR 2019; available at [openreview.net];

  • Oct. 2018 working paper (with Ioannis and Songtao) entitled “Block Alternating Optimization for Non-Convex Min-Max Problems: Algorithms and Applications in Signal Processing and Communications” has been submitted for publication, see paper [here]. ;

In this work, we consider a general block-wise one-sided non-convex minmax problem, in which the minimization problem consists of multiple blocks and is non-convex, while the maximization problem is concave. This problem arises in many SP and Comm applications, such as max-min utility maximization problem, robust learning problem, and a wireless communication problem in the presence of jammers.

Photo 

We propose two simple algorithms, which alternatingly perform one gradient descent-type step for each minimization block and one gradient ascent-type step for the maximization problem. We show that such simple alternating min-max algorithms converge to first-order stationary solutions with global sublinear rate.

  • Oct. 2018 best paper award: Haoran has received a best student paper prize (third prize) for our paper “Distributed Non-Convex First-Order Optimization and Information Processing: Lower Complexity Bounds and Rate Optimal Algorithms” in Asilomar 2018;

  • Sept. 2018 working paper (with Songtao, Xiangyi, Yongxin and Rahu) entitled “Understand the dynamics of GANs via Primal-Dual Optimization” has been submitted for publication;

  • Sept. 2018 paper accepted: our work (with Zhuoran, Kaiqing and Tamer) entitled “A Finite Sample Analysis of the Actor-Critic Algorithm” has been accepted by CDC 2018; available at [arXiv.org];

  • Sept. 2018 paper accepted: our work (with Charilaos, Xiao, Nikos) entitled “Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data” has been accepted by TSP; available at [arXiv.org];

  • Sept. 2018 paper accepted: our work (with Hoi-To, Zhuoran, Zhaoran) entitled “Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization” has been accepted by NIPS 2018; available at [arXiv.org];

  • Aug. 2018 paper accepted: our work (with Haoran) entitled “Distributed Non-Convex First-Order Optimization and Information Processing: Lower Complexity Bounds and Rate Optimal Algorithms” has been accepted by Asilomar 2018, as one of the best student paper finalists; Full version available at [arXiv.org];

1000 

This paper discusses the fundamental performance for distributed non-convex optimization, where multiple nodes collectively optimize some non-convex function using local data. For a class of non-convex problems, we develop an “optimal” distributed algorithm called xFILTER, in the sense that it achieves the best possible convergence rate for a class of first-order distributed algorithms.

In the left figure, we illustrate the results derived in this paper (both the lower bound and xFILTER bound), by comparing them with the rates of an non-optimal D-GPDA method, and the centralized gradient descent (GD) methods (over a path graph with M nodes, to achieve certain epsilon-solution). The xFILTER can significantly reduce both # of gradient evaluations, and # of communication rounds among the distributed nodes.

  • May. 2018 paper accepted: our work (with Haoran, Xiangyi, Qingjiang, Xiao and Nikos) entitled “Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management” has been accepted by IEEE TSP; available at [arXiv.org]; Prof. Nikos Sidiropoulos (ECE at University of Virginia) gave a Keynote at ICC Workshop ‘‘Machine Learning for Communications’’ about this paper. See the slides here.

  • May. 2018 M. visited ECE at Carnegie Mellon University and gave talks on ‘‘Rate Optimal Methods for Distributed Non-Convex Optimization and Learning”; Host Prof. Jose Moura.

  • May. 2018 paper accepted: our work (with Jason and Meisam) entitled “Gradient Primal-Dual Algorithm Converges to Second-Order Stationary Solutions for Nonconvex Distributed Optimization” has accepted by ICML 2018; available at [arXiv.org];

  • Apr. 2018 Dr. Hoi To Wai from Arizona State University has visited our group for a week.

  • Apr. 2017 working paper (with Haoran) entitled “Distributed Non-Convex First-Order Optimization and Information Processing: Lower Complexity Bounds and Rate Optimal Algorithms”; available at [arXiv.org];

  • Apr. 2018 paper accepted: our work (with Xiao, Kejun, Nikos and Qingjiang) entitled “Anchor-Free Correlated Topic Modeling” has been accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence for publication;

  • Apr. 2018 M. visited WID at University Wisconsin-Madison and gave talks on ‘‘Rate Optimal Methods for Distributed Non-Convex Optimization and Learning”. Host Prof. Steve Wright.

  • Mar. 2018 paper accepted: our work (with Qingjiang) entitled “Spectral Efficiency Optimization For Millimeter Wave Multi-User MIMO Systems” has been accepted by IEEE Journal on Selected Topics in Signal Processing for publication; available at [arXiv.org];

  • Feb. 2018 Mr. Ziping Zhao from Hong Kong University of Science and Technology, Prof. Daniel Palomar’s group has joined our group as a visiting student; Welcome Ziping!

  • Feb. 2018 working paper (with Songtao and Zhengdao) entitled “On the Sublinear Convergence of Randomly Perturbed Alternating Gradient Descent to Second Order Stationary Solutions” has been submitted for publication; available at [arXiv.org];

  • Feb. 2018 working paper (with Meisam and Jason) entitled “Gradient Primal-Dual Algorithm Converges to Second-Order Stationary Solutions for Nonconvex Distributed Optimization” has submitted for publication; available at [arXiv.org];

  • Feb. 2018 M. attended 2018 Zurich Seminar and have a talk on ‘‘Learning based Approach for Non-Convex Optimization and Resource Allocation”

  • Jan. 2018, M. gave a talk in IMA data science seminar on ‘‘Distributed and Learning based Approach for Non-Convex Optimization”

  • Jan. 2018, M. , with Qing, Serhat, and Tom is organizing a Symposium for GlobalSIP 2018, entitled “Distributed Optimization and Learning over Networks”. The conference is to be held Nov 26-28th in Anaheim, CA, and the submission deadline is tentatively June 10th, 2018

  • Jan. 2018, M. , with Xiao and Nikos, is organizing a special session for SPAWC 2018, entitled “Machine Learning for Communications”. The conference is to be held June 25-28th in Kalamata, Greece

2017

  • Dec. 2017, M. visited Ken Ma’s group at CUHK and gave a talk on ‘‘Distributed and Learning based Approach for Non-Convex Optimization”

  • Dec. 2017, working two-part paper (first-part with Qingjiang, second-part with Qingjiang, Tsung-Hui and Xiao) entitled “Penalty Dual Decomposition Method For Nonsmooth Nonconvex Optimization” has submitted for publication; available at [arXiv.org];

  • Nov. 2017, Davood has successfully defended his thesis, congratulations! ; After the defense, he will join Duke University for a postdoc.

  • Nov. 2017, working paper (joint work with Davood) entitled “Perturbed Proximal Primal Dual Algorithm for Nonconvex Nonsmooth Optimization” has submitted for publication.

  • Oct. 2017, working paper (joint work with Haoran, Xiangyi, Qingjiang, Xiao and Nikos) entitled “Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management” has submitted for publication; available at [arXiv.org] [code].

  • August. 2017, paper accepted: our paper (joint work with Nan, Tsung-Hui, Ya-Feng, Hamid and Tom) entitled “Network Slicing for Service-Oriented Networks Under Resource Constraints” has been conditionally accepted by IEEE JSAC.

  • Aug. 2017, grant approved: The proposal entitled “Decomposition Framework for Non-convex Nonsmooth Optimization with Applications in Data Analytics” is funded by NSF CMMI from 2017-2021 (Grant No. CMMI-1727757). We propose to use novel primal-dual methods to deal with non-convex optimization problems arise in many modern data analytics problems.

  • Aug. 2017 I am hiring a new post-doctoral fellow in the area of optimization and/or statistical learning and/or signal and information processing. Please contact me if you are interested.

  • Aug. 2017 I am moving to the ECE Department, University of Minnesota as an Assistant Professor in Fall 2017. I am very excited to start this new position and am thankful for the wonderful experience that I had in Iowa State!

  • July. 2017, M. gave an invited talk at the Wireless Broadband Department, Huawei Research Institute, Shanghai

  • July. 2017, M. gave an invited talk at the Mathematics Department, Sichuan University

  • July. 2017, M. gave an invited talk at University of Electronic Science and Technology at China

  • June. 2017, paper accepted: our paper (joint work with Wei-Cheng, Hamid, and Tom) entitled “A Distributed Semi-Asynchronous Algorithm for Network Traffic Engineering” has been conditionally accepted by IEEE SIPN.

  • May. 2017, paper accepted: our paper (joint work with Qingjiang, Haoran, Songtao and Meisam) entitled “Inexact Block Coordinate Descent Methods For Symmetric Nonnegative Matrix Factorization” (available at [arXiv.org]) has been conditionally accepted by IEEE TSP.

  • May. 2017, M. attended [SIAM Conference on Optimization]. Our minisymposium on Large-Scale optimization and data analytics has been very successful. Thanks for all the participants!

  • May. 2017, two papers accepted: our papers entitled “Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering” (available at [arXiv.org]), and “A Proximal Primal-Dual Algorithm for Fast Distributed Nonconvex Optimization and Learning Over Networks”, are accepted by ICML 2017.

  • Apr. 2017, Songtao has been awarded a Graduate College’s Research Excellence Award, congratulations!

  • Apr. 2017, paper accepted: journal paper (joint work with Xiao, Kejun, Anthony and Nikos), entitled “Scalable and Optimal Generalized Canonical Correlation Analysis via Alternating Optimization” has been accepted by IEEE TSP; available at [arXiv.org].

  • Mar. 2017, working paper (joint work with Nan, Ya-Feng, Hamid, Tsung-Hui and Tom) entitled “Network Slicing for Service-Oriented Networks Under Resource Constraints” has been submitted for publication.

    • In this work, we consider the “network slicing” for 5G systems, which is to optimally locate the service functions in the physical network and allocate the network resources while satisfying multiple service requirements specified by the service function chains, subject to link and node capacity constraints. We propose a number of highly efficient optimization based algorithms and study their performance.

  • Mar. 2017, M. visited the ECE Department of University of Minnesota

  • Mar. 2017, working paper (joint work with Davood and Alfredo) entitled “Zeroth Order Nonconvex Multi-Agent Optimization over Networks” has been submitted for publication.

    • In this work, we propose a novel distributed algorithm (over connected graphs) for non-convex optimization, in the challenging scenario where each agent can only access the zeroth-order information (i.e., the functional values) of its local functions.

  • Mar. 2017, working paper (joint work with Haoran, Xiangyi, Qingjiang, Xiao and Nikos) entitled “Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management” has submitted for publication.

    • In this work, we propose a new learning-based approach for wireless resource management. The key idea is to treat the input and output of a resource allocation algorithm as an unknown non-linear mapping and to use a deep neural network (DNN) to approximate it. The paper is available at [arXiv.org].

  • Mar. 2017, M. visited the ECE Department of UC Davis

  • Mar. 2017 Qingjiang has completed his one-year research appointment and returned China. Thanks Qingjiang for all his contribution to our group!

  • Feb. 2017, M. visited the IE Department of the Texas A&M University

  • Feb. 2017, M. , with Dr. Qing Ling, Dr. Serhat Aybat, Dr. Tom Luo and Dr. Amir Asif is organizing a Symposium for GlobalSIP 2017, entitled “Distributed Optimization and Resource Management over Networks”. The conference is to be held Nov 14-16th in Montreal, Canada, and the submission deadline is tentatively May 15th, 2017

  • Feb. 2017, Grant approved: Our group has received an NVIDIA GPU Grant.

  • Jan. 2017, paper accepted: journal paper (joint work with Songtao and Zhengdao), entitled “A Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization: Convergence Analysis and Optimality” has been conditionally accepted by IEEE TSP.

  • Jan. 2017, paper accepted: conference paper (joint work with Songtao and Zhengdao), entitled “A Stochastic Nonconvex Splitting Method for Symmetric Nonnegative Matrix Factorization, ” has been accepted by AISTATS.

  • Jan. 2017, paper accepted: journal paper (joint work with Yijian, Emiliano, Sairaj and Zi), entitled “Distributed Controllers Seeking AC Optimal Power Flow Solutions Using ADMM” has been conditionally accepted by IEEE TSG.

  • Jan. 2017, Davood has been awarded a Graduate College’s Research Excellence Award, congratulations!

  • Jan. 2017, paper accepted: journal paper (joint work with Tsung-Hui), entitled “Stochastic Proximal Gradient Consensus Over Random Networks” has been conditionally accepted by IEEE TSP; paper available at [arXiv.org].

2016

  • Nov 2016, paper accepted: journal paper, entitled “A Distributed, Asynchronous and Incremental Algorithm for Nonconvex Optimization: An ADMM Approach” has been accepted by TCNS.

  • Nov 2016, paper accepted: journal paper (joint work with Mingmin, Yunlong, etc), entitled “Joint Transceiver Designs for Full-Duplex K-Pair MIMO Interference Channel with SWIPT” has been accepted by TCOM.

  • Nov. 2016, working paper (joint work with Qingjiang) entitled “Spectral Efficiency Optimization For mmWave Multiuser MIMO Systems”, has been submitted for publication. In this work, we propose a novel optimization based non-convex algorithm to deal with multi-user hybrid precoding problem that arise in the emerging massive MIMO systems.

  • Oct. 2016, working paper (joint work with Bo, Xiao and Nicholas) entitled “Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering”, has been available at [arXiv.org], [code].

  • Sept. 2016, submitted 4 papers to ICASSP 2016.

  • Sept. 2016, working paper (joint work with Haoran, Xiangyi, Qingjiang and Xiao) entitled “Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management” has been available at [PDF] [code]. In this work, we propose a new learning-based approach for wireless resource management. The key idea is to treat the input and output of a resource allocation algorithm as an unknown non-linear mapping and to use a deep neural network (DNN) to approximate it.

  • August 2016, M. visited Princeton University, and gave a talk in the Statistical Machine Learning Group Seminar (host: Dr. Han Liu)

  • August 2016, paper accepted: conference paper (joint work with Davood, Tuo and Zhaoran), entitled “NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization” has been accepted by NIPS 2016; available at [arXiv.org].

  • August 2016, paper accepted: conference paper (joint work with Ming-min and Qingjiang), entitled “A Distributed Algorithm For Dictionary Learning over Networks” has been accepted by GlobalSIP 2016.

  • August 2016, paper accepted: Journal paper (joint work with Qingjiang, et al), entitled “Joint Source-Relay Design for Full–Duplex MIMO AF Relay Systems”, accepted by TSP;

  • July 2016, paper accepted: Journal paper (joint work with Xiangfeng, Tom and Meisam), entitled "Iterations Complexity Analysis of Block Coordinate Descent Method”, accepted by MPA; available at [arXiv.org]

  • June 2016, working paper (joint work with Qingjiang, Haoran, Songtao and Meisam) entitled “Inexact Block Coordinate Descent Methods For Symmetric Nonnegative Matrix Factorization” has been available at [arXiv.org] [code]. In this work we design block coordinate descent based algorithms for solving symmetric non-negative matrix factorization problems, and perform theoretical analysis of convergence and rate of convergence. We also apply the derived methods to document clustering problems.

  • May 2016, working paper (joint work with Xiao, Kejun, Nikos and Anthony) entitled “Scalable and Optimal Generalized Canonical Correlation Analysis via Alternating Optimization” has been available at [arXiv.org]. This paper designs a fast, memory efficient and computationally lighweight algorithm for high-dimensional generalized (multiview) canonical correlation analysis (GCCA) for large-scale datasets. Simulations and large-scale word embedding tasks are employed to showcase the effectiveness of the proposed algorithm.

  • May 2016, working paper (joint work with Xingguo, Jarvis, Raman, Han and Tuo) entitled “A First Order Free Lunch for SQRT-Lasso” has been available at [arXiv.org]. In this paper, we study the fundamental tradeoff between computation and statistical estimation robustness through a SQRT-Lasso problem for sparse linear regression and sparse precision matrix estimation in high dimensions. We explain how novel optimization techniques help address these computational challenges.

  • May 2016, working paper (by Davood, M., Zhaoran and Tuo) entitled “NESTT: A Nonconvex Primal-Dual Splitting Method for Distributed and Stochastic Optimization” has been available at [arXiv.org]. We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of N nonconvex functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into N subproblems, and utilizes an augmented Lagrangian based primal-dual scheme to solve it in a distributed and stochastic manner. We show that the proposed algorithm achieves fast convergence, under a novel non-uniform sampling scheme. Further, we reveal a fundamental connection between primal-dual based methods and a few primal only methods such as IAG/SAG/SAGA

  • May. 2016, paper accepted: journal paper (joint work with Tom), entitled "On the Linear Convergence of the Alternating Direction Method of Multipliers”, accepted by MPA; available at [arXiv.org]

  • May 2016, M. visited IMA to attend the workshop on Resource Trade-offs: Computation, Communication, and Information

  • Apr. 2016, M. visited Purdue University (host: Dr. Aldo Scutari)

  • Apr. 2016, paper accepted: conference paper (joint work with Chao Hu et al), entitled "On-Board Analysis of Degradation Mechanisms of Lithium-Ion Battery using Differential Voltage Analysis”, accepted by ASME IDETCT conference. In this work, we study reliability of lithium-ion (Li-ion) rechargeable batteries. This work proposes a methodological framework for on-board quantitative analysis of degradation mechanisms of Li-ion battery using differential voltage analysis. Simulation results obtained from LiCoO2/graphite Li-ion cells verify the effectiveness of the proposed framework in online estimation of the degradation parameters.

  • Apr. 2016, working paper entitled “Decomposing Nonconvex Problems Using a Proximal Primal-Dual Approach: Algorithms, Convergence, and Applications” has been available at [arXiv.org]

  • Apr. 2016, M. is organizing an invited session for INFORMS 2016 under the Big Data Cluster

  • Apr. 2016, M. visited The University of Alabama and gave a talk about the analysis of Block Coordinate Descent at the Applied Math Seminar (host: Dr. Brendan Ames)

  • Mar. 2016, M. attended ICASSP 2016 in Shanghai

  • Mar. 2016, M. attended the Shenzhen Research Institute of Big Data First Workshop of Data Science

  • Mar. 2016, working paper (joint work with Qingjiang et al) entitled “Joint Source-Relay Design for Full–Duplex MIMO AF Relay Systems” has been submitted for journal publication

  • Mar. 2016, paper accepted: journal paper (joint work with Ya-Feng and Enbing), entitled "Sample Approximation-Based Deflation Approaches for Chance SINR Constrained Joint Power and Admission Control”, accepted by TWC; available at [arXiv.org]

  • Feb. 2016, M. visited Arizona State University and gave a talk in the Network Science Seminar.

  • Feb. 2016, M. , with Dr. Qing Ling, Dr. Serhat Aybat and Dr. Tom Luo is organizing a Symposium for GlobalSIP 2016, entitled “Distributed Optimization and Resource Management over Networks”.

  • Feb. 2016, Dr. Qingjiang Shi has joined the group as a Post-Doctoral Research Fellow; Welcome!

  • Jan. 2016, M. attended the IMA workshop on Optimization and Parsimonious Modeling at University of Minnesota.

  • Jan. 2016, paper accepted: journal paper (joint work with Tsung-Hui, Xiangfeng and Wei-Cheng), entitled "Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis”, accepted by TSP; available at [arXiv.org]

  • Jan. 2016, paper accepted: journal paper (joint work with Tsung-Hui, Xiangfeng and Wei-Cheng), entitled "Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis”, accepted by TSP; available at [arXiv.org]

  • Dec. 2015, paper accepted: four papers have been accepted by ICASSP 2016

  • Dec. 2015, paper accepted: conference paper (joint work with Xingguo, Tuo Zhao, Raman and Han entitled ‘‘An Improved Convergence Analysis of Cyclic Block Coordinate Descent-type Methods for Strongly Convex Minimization”, has been accepted by AISTATS 2016

  • Dec. 2015, working paper (joint work with Ruoyu) entitled “Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems” is online; available at [arXiv.org]

  • Dec. 2015, M. gave a talk at the Computer Science Department, Center for Language And Speech Processing Seminar, The Johns Hopkins University.

  • Dec., 2015, Davood gave a talk in GlobalSIP 2015, [slides]

  • Dec. 2015, M. attended NIPS 2015. Presented our work on improved iteration complexity analysis for cyclic BCD algorithm (joint work with Ruoyu); see the technical report available at [arXiv.org]

2015

  • Dec, 2015, paper accepted: four papers have been accepted by ICASSP 2016

  • Dec, 2015, paper accepted: conference paper (joint work with Xingguo, Tuo Zhao, Raman and Han entitled ‘‘An Improved Convergence Analysis of Cyclic Block Coordinate Descent-type Methods for Strongly Convex Minimization”, has been accepted by AISTATS 2016

  • Dec., 2015, working paper (joint work with Ruoyu) entitled “Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems” is online; available at [arXiv.org]

  • Dec., 2015, M. gave a talk at the Computer Science Department, Center for Language And Speech Processing Seminar, The Johns Hopkins University.

  • Dec., 2015, Davood gave a talk in GlobalSIP 2015, [slides]

  • Dec., 2015, M. attended NIPS 2015. Presented our work on improved iteration complexity analysis for cyclic BCD algorithm (joint work with Ruoyu); see the technical report available at [arXiv.org]

  • Nov., 2015, working paper (joint work with Tsung-Hui) entitled “Stochastic Proximal Gradient Consensus Over Random Networks” is online; available at [arXiv.org]

  • Nov, 2015, paper accepted: journal paper (joint work with Brendan) entitled “Alternating direction method of multipliers for penalized zero-variance discriminant analysis” has been accepted by Computational Optimization and Applications (COAP); available at [arXiv.org]

  • Nov, 2015, paper accepted: journal paper (joint work with Meisam and Tom) entitled “Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems” has been accepted by SIAM Journal on Optimization; available at [arXiv.org]

  • Oct., 2015, M. organized the session “Recent Advanced in First Order Methods for Large-Scale Optimization” at INFORMS 2015; also gave two talks

    • “Stochastic Proximal Gradient Consensus Over Time-Varying Multi-Agent Network”, [slides]

    • “Iteration Complexity Analysis of Block Coordinate Descent Method”, [slides]

  • Oct., 2015, paper accepted: journal paper (joint work with Meisam, Qingjiang and Tom) entitled SINR Constrained Beamforming for a MIMO Multi-user Downlink System" has been accepted by TSP (with minor revision); available at [arXiv.org]

  • Oct., 2015, M. gave a talk at the Computer Science Department, University of Iowa, entitled “A Unified Framework for Large-Scale Block-Structured Optimization”

  • Oct., 2015, M. gave an invited talk at Allerton 2015 on distributed and stochastic optimization over random graph

  • Oct., 2015, project started: collaborative project entitled “Distributed inverter controllers seeking reliability and economic-optimality of photovoltaic-dominant distribution systems” has been started; this is a 2-year joint project with NREL

  • Oct., 2015, paper accepted: journal paper (joint work with Ya-feng and Qiang) entitled “Decomposition by Successive Convex Approximation: A Unifying Approach for Linear Transceiver Design in Heterogeneous Networks” has been accepted by IEEE Transactions on Wireless Communication; available at [arXiv.org]

  • Sept., 2015, papers submitted to ICASSP 2016

  • Sept., 2015, Davood awarded a student Travel Fellowship from GlobalSIP ; congratulations!

  • Sept., 2015, Haoran awarded a Presidential Fellowship ; congratulations!

  • Sept., 2015, grant approved: Collaborative proposal entitled “Mechanism Design for Complex Systems: A Black-box Model Approach” is funded by AFOSR;

  • Sept., 2015, paper accepted: conference paper (joint work with Ruoyu) entitled “Improved Iteration Complexity Bounds of Cyclic Block Coordinate Descent for Convex Problems” has been accepted by NIPS

  • Aug, 2015, paper accepted: journal paper (joint work with Meisam, Tom and Jong-Shi) entitled “A Unified Framework for Large-Scale Block-Structured Optimization Involving Big Data” has been accepted by IEEE Signal Processing Magazine as a Feature Article; available at [arXiv.org]

  • Aug., 2015, grant approved: Collaborative proposal entitled “Optimal Provision of Backhaul and Radio Access Networks: A Cross-Network Approach” is funded by NSF CCF; for more details please see [here].

  • Aug, 2015, paper accepted: journal paper (joint work with Qingjiang et al) entitled “Energy Efficiency Optimization for MISO SWIPT Systems With Zero-Forcing Beamforming” has been accepted by IEEE Transactions on Signal Processing; available at [IEEE Xplore]

  • Aug., 2015, paper submitted: journal paper (joint paper with Qingjiang, Meisam and Tom) entitled “SINR Constrained Beamforming for a MIMO Multi-user Downlink System”, has been submitted for publication.

  • July, 2015, M. attended ISMP 2015 and gave a talk on nonconvex ADMM

  • Jun 2015, paper submitted: journal paper (joint work with Tsung-Hui, Xiangfeng and Wei-Cheng), entitled "Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis”,submitted for publication; available at [arXiv.org]

  • Jun 2015, paper submitted: journal paper (joint work with Tsung-Hui, Xiangfeng and Wei-Cheng), entitled “Asynchronous Distributed ADMM for Large-Scale Optimization- Part I: Algorithm and Convergence Analysis”,submitted for publication; available at [arXiv.org]

  • May, 2015, M. obtained new appointment: joint (courtesy) appointment, Department of ECpE, Iowa State University

  • May, 2015, M. presented our work on BSUM and its applications in a seminar in the Department of ECpE, Iowa State University

  • Apr., 2015, M. presented 3 papers in ICASSP 2015. See here for the slides of our work on large-scale back-haul traffic engineering.

  • Apr., 2015, paper submitted: journal paper (joint paper with Andy and Tom ) entitled “Monaural Speech Enhancement: Estimating Ideal Ratio Mask by Sparse NMF and DNN” has been submitted for publication. In this work we proposed an algorithm based on Nonnegative Matrix Factorization (NMF), or Sparse Dictionary Learning, and Deep Neural Networks (DNN) to extract features for natural speech and perform speech enhancement.

  • Mar, 2015, paper accepted: journal paper (joint work with Alfredo) entitled “Efficient Rate Allocation in Wireless Networks Under Incomplete Information” has been accepted by IEEE Transactions on Automatic Control

  • Mar., 2015, paper submitted: journal paper (joint paper with Andy, Tom and researchers in Starkey Labs.) entitled “Wiener Filtering and Sparse NMF: A New Approach for Single Channel Speech Enhancement” has been submitted for publication.

  • Mar., 2015, M. attended NSF CISE Career Workshop (Thanks ISU Engineering School for providing partial funding support)

  • Mar., 2015, paper submitted: Journal paper entitled “A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data” has been submitted for journal publication.

  • Feb., 2015, paper submitted: Journal paper entitled “Energy Efficiency Optimization For MISO SWIPT Systems With Zero-Forcing Beamforming” has been submitted for journal publication.

  • Jan., 2015, M. is awarded Faculty Foreign Travel Grant by ISU faculty senate.

  • Jan., 2015, M. started a new course: Modern Big Data Optimization

  • Jan., 2015, paper accepted: Four papers are accepted by ICASSP 2015

2014

  • Dec. 2014, attended NIPS 2015, presented a poster at the main conference and a poster at the workshop of optimization for machine learning

  • Nov 2014, gave a talk at the graduate seminar, IMSE Department at Iowa State.

  • Nov 2014, visited University of Houston ECE Department, presented our work on BSUM-M algorithm convex/nonconvex ADMM; Served as committee member for Dr. Lanchao Liu

  • Nov 2014, attended INFORMS annual meeting at San Francisco; presented our work on BSUM-M algorithm

  • Nov., 2014, paper accepted: Our paper on nonconvex ADMM has been accepted by NIPS OPT Workshop.

  • Oct., 2014, five papers submitted to ICASSP 2015: We have summarized our recent works on cloud radio networking, nonconvex ADMM, hearing aids design, speech signal processing into five ICASSP submissions:

    • Analysis of ADMM for nonconvex problems; see here

    • Applying DNN for speech data analysis; see here

    • Proposing a new sparse Gaussian mixture model for Monaural speech enhancement

    • Developing algorithm for semi-asynchronous hierarchical network optimizaiton; see here

  • Oct, 2014, paper submitted: conference paper (joint work with Tom and Meisam) entitledConvergence Analysis of Alternating Direction Method of Multipliers for a family of Nonconvex Problems” has been submitted to ICASSP; This paper (as well as its journal version to be uploaded soon), is the first work that analyzes the ADMM algorithm for nonconvex problems, without making any assumptions on the iterates. We show that for a family of consensus and sharing problems, the classical ADMM as well as many of its variants (i.e., randomized , proximal, essentially cyclic) converge to the stationary solution as long as the stepsize is chosen to be large enough (with computable bounds). Also see here for a presentation made by Tom earlier in the summer.

  • Sept., 2014, I am awarded a Black & Veatch Faculty Fellowship, by the College of Engineering, Iowa State (2014-2017)

  • Sept., 2014, paper presented: Tom presented our work on convex/nonconvex ADMM in 2014 Workshop on Optimization for Modern Computation; see here for the slides

  • Aug., 2014, New course offering: IE 312 Optimization; see here for course website; permission required.

  • Aug., 2014, I started as an Assistant Professor at Iowa State University.

  • June, 2014, paper submitted: conference paper (joint work with Meisam, Tom and Jong-Shi) entitledParallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization” has been submitted for publication. This paper develops various parallel schemes (with flexible coordinate selection rules) to solve a large number of convex and nonconvex nonsmooth problems arise in optimizing large-scale problems; iteration complexity analysis for both convex and nonconvex cases is provided.

  • June, 2014, paper submitted: conference paper (joint work with Andy and Tom) entitled “Combining Sparse NMF with Deep Neural Network: A New Classification-Based Approach for Speech Enhancement” has been submitted for publication.

  • June, 2014, paper accepted: Survey paper (joint work with Hadi, Meisam, Mazair, Ruoyu, Wei-cheng and Tom ) entitledCross-Layer Provisioning of Future Cellular Networks”, has been accepted by IEEE Signal Processing Magazine, special issue on the 5G revolution. This work outlines several theoretical and practical aspects of joint interference management and network provisioning for future cellular networks. A cross-layer optimization framework is proposed for joint user admission, user-base station association, power control, user grouping, transceiver design as well as routing and flow control.

  • May, 2014, attended SIAM conference on Optimization, 14 at San Diego; presented our work on iteration complexity analysis of BCD and the BSUM-M algorithm; see the slides

  • May, 2014, paper accepted: Journal paper (joint work with Shuai and others ) entitled “Outage Constrained Robust Secure Transmission for MISO Wiretap Channels” has been accepted by TWireless.

  • May, 2014, paper accepted: Conference paper (joint work with Maziar, Meisam and Tom) entitled “Joint Base Station Clustering and Beamformer Design for Partial Coordinated Transmission using Statistical Channel State Information” has been accepted by IEEE SPAWC; This paper proposes a partial joint transmission scheme for HetNet based on the long-term channel distribution information (CDI). The main tool used is the SSUM algorithm for stochastic nonsmooth optimization.

  • Mar., 2014, paper accepted: Journal paper (joint work with Joaquin, Alfredo and Ana) entitled “Interference Pricing Mechanism for Downlink Multicell Coordinated Beamforming” has been accepted by IEEE TCOM

  • Jan.- Mar., 2014, gave talks at Texas Tech University, University of Massachusetts, New Jersey Institute of Technology, Iowa State University and Oregon State University

  • Jan., 2014, paper submitted: Journal paper (joint work with Tsung-Hui, Xiangfeng, Shiqian, Meisam and Tom) entitledA Block Successive Upper Bound Minimization Method of Multipliers for Linearly Constrained Convex Optimization”, has been submitted for publication. This paper proposes a family of algorithms for convex linearly constrained problems, with multiple block variables. The algorithm alternates between optimizing certain approximate Augmented Lagrangian Functions and an inexact dual ascent step. The algorithm is flexible to allows both cyclic and randomized update of the block variables. Application areas include signal processing, wireless networking and smart grid provisioning.

  • Jan., 2014, paper submitted: Journal paper (joint work with Wei-Cheng, Meisam, Mazair, Ruoyu, Tom and researchers from Huawei Canada) entitled “Cross-Layer Provisioning of Future Cellular Networks”, has been submitted for publication. This work outlines several theoretical and practical aspects of joint interference management and network provisioning for future cellular networks. A cross-layer optimization framework is proposed for joint user admission, user-base station association, power control, user grouping, transceiver design as well as routing and flow control.

  • Jan. 2014, I have been promoted to Research Assistant Professor.

2013

  • Dec., 2013, paper submitted: Journal paper (joint work with Tsung-Hui and Xiangfeng) entitled “Multi-Agent Distributed Optimization via Inexact Consensus ADMM”, has been submitted to IEEE TSP. This paper proposes a scheme that is able to approximate the entire augmented Lagrangian during the ADMM iterations, which makes the ADMM algorithm very simple for problems with complicated structures. We show that in the application of distributed sparse Logistic Regression, our scheme can reduce the overall computational complexity by an order of magnitude, comparing with traditional consensus methods.

  • Dec., 2013,paper submitted: Journal paper (joint work with Wei-Cheng, Tom and researchers from Huawei Canada) entitled “Min Flow Rate Maximization for Software Defined Radio Access Networks” has been submitted to IEEE JSAC. This paper proposes an interesting large-scale parallel optimization scheme, which can efficiently route a large number of traffic while performing wireless resource allocation, for the future software defined radio access network (SD-RAN); The scheme is based on a combination of ADMM and the WMMSE algorithms

  • Nov., 2013, paper submitted: Journal paper (joint work with Ya-Feng and Qiang) entitled “Decomposition by Successive Convex Approximation: A Unifying Approach for Linear Transceiver Design in Heterogeneous Networks” has been submitted to IEEE TSP. This paper proposes a fairly general framework to study the interference management problem in the next-generation Heterogeneous networks.

  • Oct. 2013, survey article published by Academic Press: Our Survey article has been formally published in the book Academic Press Library in Signal Processing: Volume 2: Communications and Radar Signal Processing (see here for the preprint version). In this article we systematically covered the recent development for the resource management problem in interfering wireless systems, from the signal processing and optimization point of view.

  • Oct., 2013, four papers submitted to ICASSP 2014: We have summarized our recent works on cloud radio networking, smart grid systems, large-scale computation into four ICASSP submissions. Among all the interesting works, we have:

    • developed a general algorithmic framework called the Block Coordinate Descent Methods of Multipliers (BCDMM) that is able to deal with many challenging problems in signal processing and smart grid systems with over million of decision variables; see here

    • developed an algorithm for solving the demand response problem in smart grid systems with uncertain demand and volatile renewable generation; see here

  • Oct., 2013, talk at City University of Hong Kong: I presented my recent work on large-scale structured optimization in the SEEM Department, CityU, Hong Kong.

  • Oct., 2013, paper submitted: Journal paper (joint work with Xiangfeng, Meisam and Tom) entitledIteration Complexity Analysis of Block Coordinate Descent Methods” has been submitted for publication; This paper presents a general analysis of the global sublinear rate of convergence for a family of Block Coordinate Descent Type algorithms suitable for large-scale nonsmooth optimization.

  • Oct., 2013, talk at UMN, S2EuNet Workshop: I presented my work on joint base station clustering and beamformer design for Heterogeneous Networks.

  • Aug., 2013, talk at UIUC, IMSE Summer School on Multi-Agent Networked System: I presented my work on a first order method for distributedly solving large-scale SCOPs.

  • June., 2013, talk at Zhejiang University, P.R. China: I presented my work on the resource management using the technique of “decomposition by successive convex approximation” in the ISEE Department, Zhejiang University

  • May, 2013, paper accepted: Journal paper (joint work with Zi, Meisam and Tom) entitled “Joint User Grouping and Linear Virtual Beamforming: Complexity, Algorithms and Approximation Bounds” has been accepted by IEEE JSAC;

  • Apr., 2013, talk at University of Virginia: I presented my recent work on the structured optimization in the SIE Department, UVa

  • Apr., 2013, paper accepted: Journal paper (joint work with Qiang and others) entitled “Transmit Solutions for MIMO Wiretap Channels using Alternating Optimization and Water-Filling” has been accepted to IEEE JSAC; This work studies a family of algorithms for efficiently performing secured point to point transmission in the presence of multiple eavesdroppers.

  • Mar., 2013, paper submitted: Conference paper (joint work with Andy and others) entitled “A Single Channel Speech Enhancement Approach by Combining Statistical Criterion and Multi-Frame Sparse Dictionary Learning” has been submitted to InterSpeech13; This work develops an efficient sparse dictionary learning approach capable of handling very high-dimension data for speech enhancement.

  • Feb., 2013, paper accepted: Journal paper (joint work with Jorge, Stephen and Alfredo) entitled “Joint Access Point Selection and Power Allocation for Uplink Wireless Networks” has been accepted by IEEE TSP