ITradesman tradie = Tradesman.Factory(); // in reality i know it's a plumber, but in the real world you won't know who's on the other side of the tradie assignment.
tradie.Work(); // and then tradie will do the work of a plumber, or electrician etc. depending on what type of tradesman he is. The foreman doesn't need to know anything, apart from telling the anonymous tradie to get to Work()!!
和sqoop-merge --merge-key
之间的差异只是sqoop --import --merge-key
会先将增量数据保存到hdfs然后与旧数据合并,而sqoop-merge --merge-key
将加载新数据并直接将数据与旧数据合并?
如果我的旧数据太大而无法合并,哪种更新旧数据的方式更好?
现在我的旧数据大约是40GB,当我使用sqoop --import --merge-key
来更新数据时,我需要更长时间才能直接加载所有数据。由于RAM的使用,有时会导致mapreduce作业失败。那么,我可以进行一些优化来解决这个问题吗?