基於Hadoop系統的雲端應用程式特徵擷取與計算監測架構 ; A Profiling and Monitoring Framework for Cloud Applications on Hadoop System

NCU Institutional Repository > 資訊電機學院 > 資訊工程研究所 > 博碩士論文 > Item 987654321/44634

請使用永久網址來引用或連結此文件: http://ir.lib.ncu.edu.tw/handle/987654321/44634

題名:	基於Hadoop系統的雲端應用程式特徵擷取與計算監測架構;A Profiling and Monitoring Framework for Cloud Applications on Hadoop System
作者:	劉勝豪;Sheng-Hao Liu
貢獻者:	資訊工程研究所
關鍵詞:	特徵擷取;監測服務;雲端計算;Profiling;Monitoring;Hadoop;Cloud Computing
日期:	2010-07-21
上傳時間:	2010-12-09 13:51:08 (UTC+8)
出版者:	國立中央大學
摘要:	近年來由於雲端計算技術的快速發展，越來越多的領域採用雲端計算技術來協助相關的計算，並相繼開發出各種類型的應用程式。這些應用程式會佔用了大量的計算資源，因此需要透過適當的監測軟體來提供系統資訊，使該雲端系統可以據此調整計算資源的使用以提升系統的整體效能與使用者的滿意度。除此之外，如果雲端系統是屬於收費型的系統，則該系統就需要提供精確的使用者資源使用狀況，並根據使用狀況去收費。但是目前的監測系統多是單純的監測硬體資源使用狀況，例如：CPU使用率、Memory使用量、網路傳輸速率等，不足以提供雲端系統實際運作的需求。因此，我們引進了 Application-Aware的概念，在 Hadoop 系統上以現有的監測系統為基礎去發展一個應用程式特徵擷取與監控（Application Profiling and Monitoring）架構。這個系統能夠把複雜的計算工作對應關係隱藏起來，使得管理者可以用更簡單的方式去掌握應用程式。這個系統主要包含三個元件：Application-Aware Profiling Agent、Profiling Database、Filter。 Application-Aware Profiling Agent安裝在每個計算節點上以紀錄各個雲端應用程式的執行情況（例如執行時間、CPU 使用狀況等資訊），這些監測資訊將透過Filter來擷取出對應關係，並將資料送到 Profiling Database 儲存。透過這一個系統，我們就可以掌握各個雲端應用程式的執行情況。除此之外，系統提供特徵擷取的功能，因此可以對計算工作的屬性進行分類。基於使用者付費與服務品質的概念，雲端計算系統中採用越穩定的計算資源進行計算的使用者，就應付出較多的成本，當然雲端計算系統對於使用者的保障當然也要越大。因此，這個研究最大的貢獻就是提高監測的階層，以應用程式作為監測對象，提供一個監測應用程式的機制，劃分出應用程式與使用者的等級，才能讓雲端系統設計出保障各等級的使用者的方法。此外，系統處理過的資料也能夠回饋給管理者以及系統的資源調整機制，進而達到在動態的雲端計算環境中去支援動態系統調整，並保障使用者可佔用的計算資源份量。The emerging cloud computing technology provides on-demand, powerful computing platforms for many complex scientific and industrial applications. They usually consume lots of computing resources and execute concurrently on a cloud platform. Therefore, a cloud system demands a good monitoring and profiling framework to keep track of users’ applications, and uses the observed information for system management purposes, such as process deployment, application optimization, and load balancing. A pay-per-use cloud system can also charge their customers ac-cording to the observed application usage. However, existing monitoring systems focus on hardware monitoring, such as CPU usage, memory usage, and network bandwidth usage. They have no clue of how users' applications utilize the system re-sources. As a result, we introduce the concept of application-aware monitoring to improve existing cloud monitoring systems, and develop an application profiling and monitoring framework based on a cloud system, Hadoop. The proposed framework does not present low-level views of jobs, tasks, and local processes. Instead, it provides a more integrated, abstract view for cloud appli-cations. The proposed architecture is comprised of three components --- the applica-tion-aware profiling agents, filters, and the profiling database. The application-aware profiling agents are installed on every computing node to record the execution status of users’ applications. The observed information is then sent to the filters for prelim-inary processing. The filters extract the mapping relations, save the results as inter-mediate files, and deliver the files to the profiling database. In addition, our system provides a classification service that utilizes the profiling data to classify cloud ap-plications. It helps users and administrators optimize their applications. The major difference between our system and other existing systems is that our system is appli-cation-oriented, while others are mostly hardware-oriented. The major contribution of our system is that it can integrate the information of users, applications, jobs, processes, and resources. When problems arise in a cloud system, applications with high performance guarantee can be identified easily to get timely service. Cloud ser-vice providers can also take advantage of our system to develop a set of billing strat-egies, to create different service-level agreements, and to protect the rights for dif-ferent customers who pay different amount of money. Furthermore, the processed data can be sent to the load-balancing service of a cloud system to support dynamic system reconfiguration and improve resource utilization rate.
顯示於類別:	[資訊工程研究所] 博碩士論文

文件中的檔案:

檔案	描述	大小	格式	瀏覽次數
index.html		0Kb	HTML	516	檢視/開啟

在NCUIR中所有的資料項目都受到原著作權保護.

社群 sharing

資料載入中.....