- Tytuł:
- Cassiopeia – Towards a Distributed and Composable Crawling Platform
- Autorzy:
-
Siwik, L.
Marcjan, R.
Włodarczyk, K. - Powiązania:
- https://bibliotekanauki.pl/articles/308019.pdf
- Data publikacji:
- 2014
- Wydawca:
- Instytut Łączności - Państwowy Instytut Badawczy
- Tematy:
-
composable software
distributed Web crawling framework
event–driven architecture
event-driven processing
SEDA
Web crawler - Opis:
- When it comes to designing and implementing crawling systems or Internet robots, it is of the utmost importance to first address efficiency and scalability issues (from a technical and architectural point of view), due to the enormous size and unimaginable structural complexity of the World Wide Web. There are, however, a significant number of users for whom flexibility and ease of execution are as important as efficiency. Running, defining, and composing Internet robots and crawlers according to dynamically-changing requirements and use-cases in the easiest possible way (e.g. in a graphical, drag & drop manner) is necessary especially for criminal analysts. The goal of this paper is to present the idea, design, crucial architectural elements, Proof-of-Concept (PoC) implementation, and preliminary experimental assessment of Cassiopeia framework, i.e. an all-in-one studio addressing both of the above-mentioned aspects.
- Źródło:
-
Journal of Telecommunications and Information Technology; 2014, 2; 79-89
1509-4553
1899-8852 - Pojawia się w:
- Journal of Telecommunications and Information Technology
- Dostawca treści:
- Biblioteka Nauki
Artykuł