Today's Question:  What does your personal desk look like?        GIVE A SHOUT

SEARCH KEYWORD -- Distributed system

  System programming is still there

System programming is the practice of writing system software. System software lives at a low level, interfacing directly with the kernel and core system libraries. Your shell and your text editor, your compiler and your debugger, your core utilities and system daemons are all system software. But so are the network server, the web server, and the database. These components are entirely system software, primarily if not exclusively interfacing with the kernel and the C library. But nowadays more...

   System programming     2014-02-27 05:46:48

  Why to opt for Hadoop?

Hadoop is a open source that stores and processes big data. The framework is written in Java for distributed processing and distributed storage of very large data. Hadoop is Scalable. It is a scalable platform because it stores and distributed large amount of data sets to hundreds and thousands of servers that operate in parallel. Traditional database systems cannot process large amount of data. But, hadoop enable business to run applications involving thousands of Terabyte data. Hadoop is ...

       2015-09-22 10:17:43

  Hadoop or Spark: Which One is Better?

What is Hadoop? Hadoop is one of the widely used Apache-based frameworks for big data analysis. It allows distributed processing of large data set over the computer clusters. Its scalable feature leverages the power of one to thousands of system for computing and storage purpose. A complete Hadoop framework comprised of various modules such as: Hadoop Yet Another Resource Negotiator (YARN MapReduce (Distributed processing engine) Hadoop Distributed File System (HDFS) Hadoop Common Thes...

   COMPARISON,HADOOP,SPARK     2018-11-22 07:08:57

  Push docker image to remote AWS ECR

With the popularity of distributed and large-scale systems, there are more and more adoptions of cloud services. One of the most popular container in the market is Docker and one of the most popular cloud service provider is AWS. From development perspective, there is frequent need to push local docker image to remote ECR for debugging or testing purpose. Normally when a code change is done and the committed change would go through a series of process like code review, push to remote repo, merge...

   CLOUD,AWS,DOCKER,AWS ECR     2020-12-13 04:12:26

  Remote execute command in Java example

Frequently there is a need to logon to a remote system and run some commands or programs to get the output, many software can serve this purpose such as putty and gitshell.These software usually provide secure access to the remote system. But have you ever wondered what to do if you need to run commands on many different systems at the same time and get all these results back at a single place? Especially in big data era, many tasks may run on different distributed systems and you want o have a ...

   JSch, Example, Distributed system, SSH2     2014-12-12 03:20:19

  Four reasons we don’t apply the 80/20 rule

Why can’t we make more use of the 80/20 rule? I’ll review what the 80/20 rule is, explain how it can be powerful, then give four reasons why we don’t take advantage of it. What is the 80/20 rule? The 80/20 rule is amazing when you first learn about it. It says that efforts and results are often very unevenly distributed. You’ll get 80% of your results from the first 20% of your efforts. For example, maybe your top 20% of customers will provide 80% of your profit. O...

   80/20,Development,Profit,Revenue,Effort     2012-02-08 10:03:50

  Computer skills one can learn within one day

Computer related technical skills are usually thought as complicated and difficult to understand. It's very difficult for one to get hands on one skill or master one skill. But if you really do want to learn something useful within one day, there are some good choices which will not take too long to get to know and use.. Version control:- Git, GitHub and SVN Regular expressions AWK sed Grep Learn how to do things with Vim that you never knew could be done. Set up a crawler that can scrape some ...

   Computer skills,Learn     2013-09-16 08:51:35

  Redis Cluster and Common Partition Techniques in Distributed Cache

In this post, I will discuss a few common partition techniques in distributed cache. Especially, I will elaborate on my understanding on the use of Redis Cluster. Please understand that at the time of writing, the latest version of Redis is 4.0.10. Many articles on the same topic have a different idea from this post. This is mainly because, those articles are probably outdated. In particular, they may refer to the Redis Cluster implementation in Redis 3. Redis Cluster has been improved...

   REDIS,DISTRIBUTED CACHE,CLOUD COMPUTING     2019-01-17 08:10:37

  etcd installation and usage

etcd is an open source and highly available distributed key-value storage system and is commonly used in critical data storage and service discovery and registration use cases. It is focusing on: Simple: well-defined, user-facing API (gRPC) Secure: automatic TLS with optional client cert authentication Fast: benchmarked 10,000 writes/sec Reliable: properly distributed using Raft etcd and Redis both support key-value storage and can be set up in distributed systems. Also Redis supporst more key...

   ETCD,TUTORIAL,RAFT,DISTRIBUTED SYSTEM     2021-03-07 03:10:33

  What is Hystrix and How does Hystrix work

Background In distributed systems, there is one effect where the unavailability of one service or some services will lead to the service unavailability of the whole system, this is called service avalanche effect. A common way to prevent service avalanche is do manual service fallback, in fact Hystrix also provides another option beside this. Definition of Service Avalanche Effect Service avalanche effect is a kind of effect where the service provider fails to provide service which causes t...

   AVALANCHE EFFECT,HYSTRIX,DISTRIBUTED SYSTEM     2019-02-04 06:00:38

RECENT