gist JS

Friday, August 16, 2013

hbase scan: batch vs cache

Here's today's contribution to the Internet: tl;dr When it comes to HBase scanner settings, you want caching, not batchsize. Maybe this is totally clear to everyone else. But for those of us who are 'newer to hbase' I can never quite remember what I'm doing.
Say you've got this code:
Scan s = new Scan(startKey);s.setCaching(foo);s.setBatch(bar);ResultScanner scanner = new ResultScanner(s);for (final Result r : scanner) {  //stuff}
But you're clever and you don't want to do RPC calls to HBase for every row. You might even say you'd like to 'batch' the results from your scanner. 

So you read http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/Scan.html public void setBatch(int batch)
Set the maximum number of values to return for each call to next()
public void setCaching(int caching)
Set the number of rows for caching that will be passed to scanners. If not set, the default setting from HTable.getScannerCaching() will apply. Higher caching values will enable faster scanners but will use more memory.
Annnd.... not sure. I mean, I only want one Result every time I call next()in my iterator, right? What would  a number >1 even mean? 

And I'm sure I shouldn't set 'caching' that sounds like it will 'cache' something. I want to read the real stuff.

But you do want caching. Caching is how many things come back in a batch from your scanner. 

Ok. Fine. Caching got named poorly.  What is batch?

Batch is in case you have super wide rows. Say you have 250 columns. Batch of 100 would give your iterator:
  • Iteration 1: Result id 0. Columns 0-99
  • Iteration 2: Result id 0. Columns 100-199
  • Iteration 3: Result id 0. Columns 200-249
  • Iteration 4: Result id 1. Columns 0-99
  • Iteration 5: Result id 1. Columns 100-199
Or at least that's what http://twitter.com/monkeyatlarge told me.


Anonymous said...

Does this apply for HBase REST as well?

The caching seems to not work using REST or probably me doing it wrong.

Jeff Dwyer said...

Haven't used the REST stuff myself. I presume you've seen https://issues.apache.org/jira/browse/HBASE-7803 ? Seems as though it is supposed to work.

Dan Häberlein said...

Good stuff!

angela michael said...

commission express reviews I really like and appreciate your blog post.Much thanks again. Want more.

Govt Jobs said...

I am one of those many candidates who completely rely on this website to know about government jobs. Thanks a lot providing exams syllabus along with good material to easily prepare.

Brigade Buena Vista said...

Nice Blog. Useful Information.Thanks for sharing with us.
Brigade Buena Vista
Lodha bellezza Hyderabad
Godrej Avenues Bangalore
Goyal Orchid Whitefield

Mark Smith said...

Rolex Watches Authentic Mens & Ladies Rolex Datejust, President Watches for Sale at JavyEstrella.com.

Amlon Terence said...

Florida Accident Lawyer Sinclair Law - Leading Florida Law Firm Attorney Brad Sinclair Represents Personal Injury, Car Accident, Motorcycle Accident Victims in Melbourne.

Saradha Devi said...

interesting article to read
Selenium Training | Selenium Training Institute in Chennai | Best Selenium Training Institutes in Chennai | Software Testing Training in Chennai

jessie karthik said...

JNTUK 2-1 Results Check

Manasa Garugu said...

Sarkari Recruitment is one of the biggest Indian Job Site so here you will getPPSC Jobs 2017 so

Creative Template said...

nice post.
Free Wordpress Themes

proudstories said...

thanks for sharing information.
quotes by famous people

Srinu said...

Nice blog and thanks for sharing the post. Check out some latest projects.

Goyal Orchid Whitefield
Prestige Lakeside Habitat
Rainbow Vistas Rock Garden
Prestige Ivy League
Lodha Meridian
Lodha Bellezza Sky Villas
Prestige Kew Gardens

Rajapriya R said...

great blog to read.. after reading this blog i learned more useful information from this blog

hadoop training institute in velachery | big data training institute in velachery | hadoop training in chennai velachery | big data training in chennai velachery

Ananthi S said...

this blog really helpful to develop my career in Hadoop.. thank you for sharing

hadoop training institute in adyar | big data training institute in adyar | hadoop training in chennai adyar | big data training in chennai adyar

Sujit kUmar said...

nice post for sharing Selenium article. its really helpful for me. keep sharing tutorials????????????????

Rajesh Kumar K said...

I just came across from this article and I read it about some tutorial on selenium related. It is too good and worth reading thanks for sharing

kingrani said...

Click here to Get UPSC IFS Admit Card
Apply Online for DPMU Jamshedpur Recruitment

Mirnalini Sathya said...

The way you have explained the Big Data concept is really superb! I have never come across such a informative blog in my career. Nowadays, Hadoop course is in high demand by most of the professional to enhance their career.


Big Data Training |
Big Data Course in Chennai

Suba said...

Nice blog on Selenium Testing!!! I agree with your points, Testing will help whole organization to handle the projects in efficient manner.

Selenium Training |
selenium training in T Nagar 

sophia sophia said...

Thanks for this wonderful post. The information in this article is very helpful to me. Thanks a lot for sharing. Keep blogging.
Interest Waypoint
Best Truth or Dare Questions for Kids

aashutosh webtrackker said...

Aws online training in india
Salesforce online training in india

SAS Online Training in india

Salesforce admin online training in india

Linux Online training in India

React.JS online training in india

aashutosh webtrackker said...

Python online Training in India

Oracle DBA online training in India

Java online Training in India

SAP online Training In india

Digital Marketing Online Training in India

aashutosh webtrackker said...

Aws online training in india
Salesforce online training in india

SAS Online Training in india

Salesforce admin online training in india

Linux Online training in India

React.JS online training in india

Srinu said...

Nice blog and thanks for sharing the post. Check out some latest projects.

Provident Kenworth
SMR Vinay Iconia
Provident The Pearl
Prestige Ivy League
Western Exotica
NCC Urban Gardenia
Quiescent Heights
NCC Urban One
NorthStar Homes District1
Provident Kenworth Hyderabad

ASO Services said...

There are so many useful information in your post, I like it all the time. And I want to share this boost app ranking with you, you can take a look. This is very useful.

Richard Swayar said...

Australia Best Tutor is one of the best Online Assignment Help providers at an affordable price. Here All Learners or Students are getting best quality assignment help with reference and styles formatting.

Visit us for more Information

Australia Best Tutor
Sydney, NSW, Australia
Call @ +61-730-407-305
Live Chat @ https://www.australiabesttutor.com

Our Services

Online assignment help
my assignment help Student
Assignment help Student
help with assignment Student
Students instant assignment help
Students Assignment help Services

priyanka sria said...

create android app without coding

Srinu said...

Nice blog and thanks for sharing the post.

Salarpuria Sattva Magnus
Sumadhura Acropolis
SMR Vinay Iconia
Prestige Ivy League
Jayabheri The Peak
Muppas Indraprastha
Patel Smondo
Vertex Panache
Western Exotica
NCC Urban Gardenia

priyanka sria said...

fax through gmail

Cricket Fan said...

Ramadan fasting greetings 2018

Mahesh said...

Really very informative and creative contents. This concept is a good way to enhance the knowledge.thanks for sharing.
please keep it up.
selenium training in gurgaon

tansitanu said...

Thanks a lot very much for the high quality and results-oriented help. I won’t think twice to endorse your blog post to anybody who wants and needs support about this area.
Best Hadoop Training Institute In chennai

Cricket Fan said...

Top 5 Best after effects plugins list

priyanka sria said...

Animated Movies

Cricket Fan said...

Create Your Own App for Free - Stunning Features & Design
3 Step Drag & Drop DIY App Maker - No Coding Needed - Create Your App Now!

Cricket Fan said...

How Webmail send and receive mail online

Cricket Fan said...

How to draw and learn drawing

AutoCAD training in Noida said...

Each department of CAD have specific programmes which, while completed could provide you with a recognisable qualification that could assist you get a job in anything design enterprise which you would really like.

AutoCAD training in Noida

AutoCAD training institute in Noida

Best AutoCAD training institute in Noida

tanmay chaubey said...

If you want to join then you must join webtrackker technology which provides 100% placement.

servicenow training institute in Noida

ServiceNow portal Training in Noida

ServiceNow integration Training in Noida

ServiceNow implementation Training in Noida

servicenow scripting Training in Noida

Best servicenow admin training institutes in Noida

Best Servicenow Developer training training institute in Noida

Unique Residency said...

Nice blog and thanks for sharing the post.

Unique Residency
Unique Residency Manikonda
Unique Residency Hyderabad

tushar rana said...

6-week summer course in Noida - 6 weeks The summer course plays a crucial role in shaping the career of young aspiring / informatics students. This training has been specifically introduced so that students can become familiar with current industrial culture and industrial needs. Webtrackker technology offers a 6-month training program for students / graduates that includes small and large projects.
6-week summer course in Noida

Cricket Fan said...

best animated movies of all time

Cricket Fan said...

Happy Diwali Greetings for Family

raman rajput said...

hotels coupon codes

hotel offers & deals

hotels discount offers

nearbuy coupon codes

nearbuy promo codes

nearbuy deals and offers

nearbuy discounts

zoomcar promo code

zoomcar coupon code

zoomcar offers on ride

zoomcar deals and offers

raman rajput said...

healthkart deals and offers

healthkart discount offers

bigbasket promo codes

bigbasket coupon codes

bigbasket offers

bigbasket coupon and deals

pizzahut promo code

pizzahut coupon codes

pizzahut offers

pizzahut coupon and offers

hotels promo code

raman rajput said...

cleartrip coupon code

cleartrip offers and deals

cleartrip deals

MMT promo Codes

MMT coupon codes

Makemytrip promo codes

makemytrip offers

makemytrip deals & offers

healthkart coupon code

healthkart promo codes

raman rajput said...

Flipkart promo codes

Flipkart deals & coupons

flipkart coupon code

flipkart coupons offer promo code

Amazon promo code

amazon offers

amazon offers and deals

amazon coupon code

amazon deal of the day

cleartrip promo codes

raman rajput said...

Myntra promo codes

Myntra deals and offers

Myntra coupon codes

Myntra coupons offers promo codes

Myntra offers on online shopping

Nykaa Promo Codes

Nykaa Deals and offers

Nykaa Coupons codes

Nykaa coupons offers promo codes

Nykaa offers on online shopping

raman rajput said...

Ajio Promo Codes

Ajio Coupons & Offers

Ajio Coupon codes

Ajio Offers Promo Codes

Ajio Offers on online shopping

Firstcry Promo Codes

Firstcry Deals & offers

Firstcry coupons codes

Firstcry Coupons Offers Promo Codes

Firstcry Offers on Kids shopping

priyanka sria said...

Happy Diwali Greetings for Family

Cricket Fan said...

happy new year for friends 2019

diwali status video for whatsapp 2018

Happy dipawali images 2018

Rohan kumar said...

Cloud Computing Training In Noida
Webtrackker is IT based company in many countries. Webtrackker will provide you a real time projects based training on Cloud Computing. If you are looking for the Cloud computing training in Noida then you can join the webtrackker technology.
Cloud Computing Training In Noida , Cloud Computing Training center In Noida , Cloud Computing Training institute In Noida ,

Company Address:
Webtrackker Technology
C- 67, Sector- 63, Noida
Email: info@webtrackker.com
Website: www.webtrackker.com

prem said...

Video editing course in Noida
Video editing training institute in Noida- Webtrackker Technology is and IT Training institute providing the Video editing course in Noida, FCP, Final Cut Pro Training in Noida. For more call us- 8802820025.
Video editing course in Noida
best video editig course in noida
best video edtitng institute in noida
Company Address:
Webtrackker Technology
C- 67, Sector- 63, Noida
Phone: 01204330760, 8802820025
Email: info@webtrackker.com
Website: http://webtrackker.com/Best-training-institute-Video-editing-FCP-course-in-Noida.php

Srinu said...

Nice blog and thanks for sharing the post.

Unique Residency
SMR Vinay Iconia
Prestige Ivy League
Salarpuria Sattva Magnus
Muppas Indraprastha
Western Exotica
Urban Life Villas
Udaya Cresent
Jubilee Ballpark
NCC Urban Gardenia

johnsy sai said...

Existing without the answers to the difficulties you’ve sorted out through this guide is a critical case, as well as the kind which could have badly affected my entire career if I had not discovered your website.
Digital Marketing online training

full stack developer training in pune

full stack developer training in annanagar

full stack developer training in tambaram

full stack developer training in velachery

Cricket Fan said...

best fonts for websites

after effects templates free

List of upcoming marvel movies

Top animated movies on Netflix 2018

Mouni yoga said...

I know you feel more happy when you get things done and best of all those things are your most precious treasure.
python training in tambaram
python training in annanagar
python training in velachery

bhavna webtrackker said...

rpa training institute in noida

Blockchain training institute in Noida

amarraj paindit said...

rpa training institute in noida
sas training institute in noida
hadoop training institute in noida
blokchain traninig institute noida

aruna raj said...

This blog is the general information for the feature. You got a good work for these blog.We have a developing our creative content of this mind.Thank you for this blog. This for very interesting and useful.
Blueprism online training

Blue Prism Training in Pune

Blueprism training in tambaram

amarraj paindit said...

servicenow scripting Training in Noida
rpa training institute in noida
sas training institute in noida
hadoop training institute in noida
blokchain traninig institute noida

Naga Manickam said...

I would assume that we use more than the eyes to gauge a person's feelings. Mouth. Body language. Even voice. You could at least have given us a face in this test.
Data Science training in rajaji nagar
Data Science training in chennai
Data Science training in electronic city
Data Science training in USA
Data science training in pune
Data science training in kalyan nagar

nancy said...

Your blog is very useful for me, Thanks for your sharing.

MSBI Training in Hyderabad

Sri Nithya said...

Wonderful post. Thanks for taking time to share this information with us.
Blue Prism Training Institute in Chennai | Blue Prism course in Chennai | Blue prism course | UiPath Course | UiPath Training Institutes in Chennai | UiPath Courses in Chennai