Amazon EMR now supports Apache Iceberg, a highly performant, concurrent, ACID-compliant table format for data lakes - devamazonaws.blogspot.com

We are excited to announce that Amazon EMR 6.5.0 now includes Apache Iceberg version 0.12. Apache Iceberg is an open table format for large data sets in Amazon S3 and provides fast query performance over large tables, atomic commits, concurrent writes, and SQL-compatible table evolution. With the current release, you can use Apache Spark 3.1.2 on EMR clusters with the Iceberg table format.

Post Updated on January 21, 2022 at 07:56PM

Comments

Post a Comment

Popular posts from this blog

[MS] Pulling a single item from a C++ parameter pack by its index, remarks - devamazonaws.blogspot.com

In my earlier discussion of pulling a single item from a C++ parameter pack by its index , I noted that with the Pack Indexing proposal, you would be able to write this to pull an item from a parameter pack while preserving its reference category. template<int index, typename...Args> void example(Args&&... args) { auto&& arg = (Args...[index]&&)args...[index]; using Arg = Args...[index]&&; } Why did I need to apply the (Args...[index]&&) cast ? Why can't I just write this: auto&& arg = args...[index]; or possibly decltype(auto) arg = args...[index]; Well, when you write the name of a variable, the result is an lvalue reference, even if the variable is an rvalue reference . Watch: struct S { S(S&); // construct from lvalue S(S&&); // construct from rvalue }; void whathappens(S&& s) { S t = s; // which will it use? } Try it out in your favorite compiler. This code construc...
Read more

[MS] Debugger breakpoints are usually implemented by patching the in-memory copy of the code - devamazonaws.blogspot.com

In practice, when you set a code breakpoint in the debugger, the debugger replaces the instruction at that location with a breakpoint instruction.¹ When execution reaches that instruction, it will encounter the breakpoint instruction and break into the debugger. When the program has been stopped in the debugger, what happens next can vary from debugger to debugger. Some debuggers remove all their breakpoints when the program stops, and then restore the breakpoints when the program resumes. Other debuggers leave the breakpoints in place even when the program is stopped. In both cases, if you inspect the memory in the debugger, you will see the original unpatched code. In the first case, it's because the code really is unpatched; the breakpoint instructions are removed. In the second case, it's because the debugger is lying to you and showing you the original bytes even though they aren't what are in memory right now. Most of the time, this deception is insignificant. Eve...
Read more

[MS] The case of the crash when destructing a std::map - devamazonaws.blogspot.com

A customer reported that they were getting crashes while destructing a std::map . Here's the point of the crash: eax=245bd25c ebx=00004d33 ecx=31feece4 edx=00c40000 esi=00000000 edi=31feece4 eip=563397db esp=245bd250 ebp=245bd268 iopl=0 nv up ei ng nz na pe nc cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010286 contoso!std::_Tree<⟦...⟧>::~_Tree+0x2b: 563397db cmp byte ptr [esi+0Dh],0 ds:002b:0000000d=?? 0:054> k9 ChildEBP RetAddr (Inline) -------- contoso!std::_Tree_val<⟦...⟧>::_Erase_tree [xtree @ 1073] (Inline) -------- contoso!std::_Tree_val<⟦...⟧>::_Erase_head+0x5 [xtree @ 1073] 245bd268 56338313 contoso!std::_Tree<⟦...⟧>::~_Tree+0x2b [xtree @ 1073] 245bd290 56362695 contoso!Contoso::IdCollection::~IdCollection+0x163 245bd2b4 564c8510 contoso!Contoso::LogEntry::~LogEntry+0xd5 245bd2e0 5691a999 contoso!Contoso::LogEntries::Cleanup+0x90 (Inline) -------- con...
Read more