Handling Large Datasets in Parallel Metaheuristics: A Spares Management p Case Study y and Optimization
Chee Shin Yeo, Elaine Wong Kay Li, Yong Siang Foo
Metaheuristics
2
Solve optimization problems in diverse domains Search over a solution space for an optimal solution that will minimise an objective function Challenges Exponentially increasing execution time Memory intensive Inconsistent performance due to random generation
Parallel Metaheuristics
3
Search in parallel using multiple searches over a solution space Solution traces to reduce searching Keeping track of past searches Cooperative methods with different initial solutions Parallel searches exchange intermediate results Problem: Large datasets Insufficient Memory Network bottlenecks
Parallel Metaheuristics: Flow Control Workflow
4
Run as a flow control workflow with n states, states each state with x Multiple Independent Runs (MIR)
Optimal solution is the output On of the last state (Sn)
Flow Control Workflow: Clustering Policy Workflow Clustering Policy: Executes the entire workflow as a single job
S1 MIR1a MIR1b
Processor P2 P1
Td
S2 MIR2a MIR2b
S3 MIR3a MIR3b
entire workflow
Ta
Time
State Clustering Policy: Executes each state of the entire workflow as a single g jjob
S1 MIR1a MIR1b
Processor P2 P1
Td
S1
S2 MIR2a MIR2b
Ta Td
S2
S3 MIR3a MIR3b
Ta Td
S3
Ta
Time
Job Clustering Policy: Executes each MIR in a state as a single job
S1 MIR1a MIR1b
5
Processor P2 P1
S2 MIR2a MIR2b
S3 MIR3a MIR3b
Td MIR1a Ta Td MIR2a Ta Td MIR3a Ta Td MIR1b Ta Td MIR2b Ta Td MIR3b Ta
Time
Case Study: Spares Management and Optimization (SMO)
6
Optimization scenario Aircraft spare parts 59 airports – with time time-based based delivery commitment at selective airports Logistics flights between all locations Flight network – ~320,000 Flight Hours/year
Experimental Setup
7
Effect of Stop Criterion on IBM (2 (2.26GHz, 26GHz 32GB)
8
Effect of Clustering Policy on DELL (3 (3.0GHz, 0GHz 4GB)
9
Conclusion
10
Flow control workflow for p parallel metaheuristics Stop Criterion: Exchange of intermediate data Clustering g Policy: y Assignment g of jjobs Memory y availability y is a critical issue Less iterations for stop criterion is better Less memory, shorter completion time Job clustering policy is better Least memory, memory more reliable completion
Future Work
11
More intelligent g optimization p Self-configuring stop criterion Resource contention in multi-user environment Effective scheduling g mechanism
End of Presentation Thank You Any Questions/Comments?
Handling Large Datasets in Parallel Metaheuristics: A ...
solution space. â« Solution traces to reduce searching. â« Keeping track of past searches. â« Keeping track of past searches. â« Cooperative methods with different initial solutions. â« Cooperative methods with different initial solutions. â« Parallel searches exchange intermediate results. â« Problem: Large datasets. Insufficient ...
Feb 13, 2009 - An effective data mining tool for gene expression microarray data is to infer ..... In addition to BEST, we also tested well-established query tools ...
Feb 13, 2009 - In microarray gene expression data analysis, it is often of interest to identify ... highly desirable to develop a query tool that can automatically.
processor models. We describe the design of the simulator, provide performance ... The use of simulation, however, can aid both in their efforts to obtain high utilization from ...... A Hybrid MPI Simulator, IEEE International Conference on Cluster.
First, based on a large corpus of English-Chinese comparable patents, more than 22 million bilingual .... companies may be interested in monitoring and analyzing the patents filed in ... translation engines and more parallel data to help us.
HPPNetSim is designed to simulate large/ultra-large interconnection networks and study the communication behavior of parallel applications. In the full system simulator HPPSim, network is abstracted as a single black-box switch, which simulates commu
Jan 19, 1999 - ... less idealistic programmers can write in C, Java, Ada and other useful .... since the Prelude is not just ordinary Haskell code, requires a lot of ...
building many applications, such as machine translation (MT) and cross-lingual information retrieval. ... For statistical machine translation (SMT), tremendous strides have been made in two decades, including Brown .... candidates by learning an IBM
Apr 23, 2015 - Borg provides three main benefits: it (1) hides the details of resource ... web search, and for internal infrastructure services (e.g.,. BigTable). ... the high-performance datacenter-scale network fabric that connects them. A cluster
tools for interactive statistical analysis using this infrastructure has lagged. ... Split-apply-combine [26] is a common strategy for data analysis in R. The strategy.
Apr 23, 2015 - triggered a software defect in Borg so it can be debugged); fixing it by .... Normally, though, an online schedul- ing pass ..... memory-accounting.
Multilingual data are critical resources for building many applications, such as machine translation (MT) and cross-lingual information retrieval. Many parallel ...
Apr 21, 2008 - Spectral Clustering, Parallel Computing, Social Network. 1. INTRODUCTION .... j=1 Sij is the degree of vertex xi [5]. Consider the ..... p ) for each computer and the computation .... enough machines to do the job. On datasets ...
1 Department of ECE, UCSB. 2 Department of ... Apr. 22, 2008. Gengxin Miao Et al. (). Apr. 22, 2008. 1 / 20 .... Orkut is an Internet social network service run by.
AbstractâAn effective video copy detection framework should be robust against ... from other web sites. ..... video database about every 6.43 frames on average.
of presentation. Like its predecessor, this book will be the standard text on the subject for some. time. Journal of Chemometrics. The authors are to congratulated ...
network design, protein alignment, and many other fields of utmost economic, indus- trial and .... a B&B based system for job-shop scheduling is described.
Department of Education. National Capital Region. SCHOOLS DIVISION OFFICE. Nueva Ecija St., Bago Bantay, Quezon City. LIST OF NEWLY APPOINTED SENIOR HIGH SCHOOL TEACHERS. 'ffi,. Name of Teacher School Assigned. SAN FRANCISCO HS. 2. CABIAO, JOYCE PUGA
